# Encore documentation > The full Encore docs as markdown, in sidebar order and grouped by SDK language. Each page starts with a link to its web version and its docs section, for example "Building (Encore.ts)". The index is at https://encore.dev/llms.txt. --- Source: https://encore.dev/docs Section: Getting started # What is Encore? > Encore combines an open source SDK, CLI, and cloud platform into one workflow for building and deploying apps. Import what you need from the SDK directly into your application to define services and infrastructure. Run your app and infrastructure locally with the CLI to test changes end to end. Then push to your repo, and the platform sets up the matching resources in your cloud and deploys your app. **Start with the Quickstart** Build, run, and deploy your first app. - [Start the Quickstart guide](https://encore.dev/docs/quick-start) ## Infrastructure from your code Encore's SDK is available in multiple languages. In TypeScript, import `SQLDatabase` from `encore.dev/storage/sqldb` to query your database. Encore provides PostgreSQL locally during development. When you deploy, the Encore platform [provisions a managed database](https://encore.dev/docs/platform/infrastructure/infra) based on your cloud and environment settings: Amazon RDS or Aurora on AWS, Cloud SQL on Google Cloud, or Neon. The SDK also provides Pub/Sub, object storage, caches, cron jobs, and secrets. Encore automatically sets up access for each service. Configure capacity, regions, and backups for each environment in the platform. See [Understanding Encore](https://encore.dev/docs/understanding-encore). ## From local development to production Run `encore run` to start your application and its local infrastructure. The [development dashboard](https://encore.dev/docs/dev-dash) lets you call APIs, inspect database contents, and follow requests through [distributed traces](https://encore.dev/docs/tracing). For cloud deployment, you can start with a hosted development environment, then connect your own [AWS or GCP account](https://encore.dev/docs/platform/deploy/own-cloud) for production. See [Deployment](https://encore.dev/docs/platform/deploy/deploying). You can also deploy without using the platform: [build a Docker image](https://encore.dev/docs/self-host/build), provision your infrastructure, and supply its connection details through an [infrastructure configuration file](https://encore.dev/docs/self-host/configure-infra). ## Next steps - [Quickstart](https://encore.dev/docs/quick-start): Build and deploy your first app. - [Understanding Encore](https://encore.dev/docs/understanding-encore): How code becomes infrastructure. - [Encore.ts](https://encore.dev/docs/ts) and [Encore.go](https://encore.dev/docs/go) SDKs: APIs, databases, messaging, and other building blocks. - [Platform overview](https://encore.dev/docs/platform/introduction): How the platform manages infrastructure, environments, and deployments. - [Deployment](https://encore.dev/docs/platform/deploy/deploying): Environments, cloud accounts, and CI/CD. --- Source: https://encore.dev/docs/quick-start Section: Getting started # Quickstart Guide > Build an app in 5 minutes with Encore Let's build something small with a surprising amount of real infrastructure. You'll create a PostgreSQL-backed URL shortener, with the database provisioned and every request traced automatically. Then grow it into a multi-service app by adding a second service, its own database, and a Pub/Sub topic in code, run it all locally, and deploy it to the cloud. This page covers both Encore.ts and Encore.go. Sections that differ are labelled **Encore.ts:** and **Encore.go:**. ## Before you start You'll need: - **Encore.ts:** **[Node.js](https://nodejs.org/en/download)****Encore.go:** **[Go](https://go.dev/dl/)** - **[Docker](https://docker.com)**. Make sure it's running before step 2 so Encore can provision PostgreSQL for you locally. ## 1. Install Encore and create your app Install the Encore CLI and create an app from the URL shortener starter: **Encore.ts:** **Install the Encore CLI** (pick the line for your OS): ```shell # macOS brew install encoredev/tap/encore # Windows iwr https://encore.dev/install.ps1 | iex # Linux curl -L https://encore.dev/install.sh | bash ``` Then scaffold an app: ```shell encore app create --example=ts/url-shortener ``` Prefer not to install anything? [Deploy this example from your browser](https://app.encore.dev/create-app/clone/ts-hello-world?utm_source=encore_dev&utm_medium=deploy_button&utm_campaign=ts-hello-world&utm_content=quick-start-under-install) **Encore.go:** **Install the Encore CLI** (pick the line for your OS): ```shell # macOS brew install encoredev/tap/encore # Windows iwr https://encore.dev/install.ps1 | iex # Linux curl -L https://encore.dev/install.sh | bash ``` Then scaffold an app: ```shell encore app create --example=url-shortener ``` Prefer not to install anything? [Deploy this example from your browser](https://app.encore.dev/create-app/clone/hello-world?utm_source=encore_dev&utm_medium=deploy_button&utm_campaign=hello-world&utm_content=quick-start-under-install) If you're new to Encore, the CLI asks you to create a free account, which you'll need to deploy the app and manage secrets using Encore's cloud platform. It then offers to install [Encore's AI instructions](https://encore.dev/docs/ai-integration), letting you complete the rest of this guide using simple prompts in your AI coding tool. Finally, choose a name for your app. The CLI creates a directory with that name. If it then offers to run your app, you can answer **n**: you'll start it in step 2. ### Want to start from scratch instead? Run `encore app create` without flags and choose a template. `Hello World` is the smallest option, but the rest of this guide uses the URL shortener. ### Install Encore skills The [Encore skills package](https://github.com/encoredev/skills) supports Cursor, Claude Code, GitHub Copilot, and other agents: ```bash npx add-skill encoredev/skills ``` ## 2. Run your app Make sure Docker is running, then start your app: ```shell $ cd your-app-name # replace with the app name you picked $ encore run ``` ```output ✔ Building Encore application graph... Done! ✔ Analyzing service topology... Done! ✔ Creating PostgreSQL database cluster... Done! ✔ Running database migrations... Done! ✔ Starting Encore application... Done! Encore development server running! Your API is running at: http://127.0.0.1:4000 Development Dashboard URL: http://127.0.0.1:9400/your-app-name MCP SSE URL: http://127.0.0.1:9900/sse?app=your-app-name ``` Encore automatically provisions the PostgreSQL database and runs its migrations. ### Open the local dashboard Open [http://localhost:9400](http://localhost:9400) in your browser. The [local dashboard](https://encore.dev/docs/dev-dash) includes an API explorer, generated documentation, a database explorer, and [distributed tracing](https://encore.dev/docs/tracing). The [architecture diagram](https://encore.dev/docs/architecture) gives you a live map of your service architecture. ## 3. Call your API In the API Explorer, call `url.shorten`. Encore generates the request form from your code, so it's ready to use: ![Calling url.shorten from the API Explorer](https://encore.dev/assets/docs/qs_api_explorer.png) Each call appears in the **Traces** column on the right. Call it a few more times with different URLs, each one adding another row to the database. Then call `url.get` with one of the returned ids to fetch a URL back. ### Prefer the terminal? **Encore.ts:** ```shell $ curl -X POST http://localhost:4000/url \ -H "Content-Type: application/json" \ -d '{"url": "https://encore.dev"}' {"url": "https://encore.dev", "id": "5cJpBVRp"} $ curl http://localhost:4000/url/5cJpBVRp {"url": "https://encore.dev", "id": "5cJpBVRp"} ``` **Encore.go:** ```shell $ curl -X POST http://localhost:4000/url \ -H "Content-Type: application/json" \ -d '{"url": "https://encore.dev"}' { "ID": "5cJpBVRp", "URL": "https://encore.dev" } $ curl http://localhost:4000/url/5cJpBVRp { "ID": "5cJpBVRp", "URL": "https://encore.dev" } ``` ## 4. Look at a trace In the local dashboard, select a `POST /url` request from the right-hand column. ![Trace of a url.shorten request in the Local Dashboard](https://encore.dev/assets/docs/qs_trace_db.png) Encore captures a distributed trace for every request, with no logging or instrumentation code required. Each trace shows: - **Summary:** duration, API calls, database queries, published messages, and logs. - **Payloads:** the full request and response. - **Timeline:** where the request spent its time. - **Logs:** every log line correlated with that request. Compare it with a `GET /url/:id` trace. The database operation is now a `SELECT` instead of an `INSERT`. ### See a failure in a trace Traces are most useful when something breaks. Call `url.get` with an id that doesn't exist, such as `does-not-exist`. Open the failed request's trace. It shows the `SELECT` that returned no rows and the error returned by the endpoint. **Encore.ts:** The endpoint throws `APIError.notFound`. Encore returns a `404` and records the error in the trace. **Encore.go:** The database returns `sql.ErrNoRows`, which the endpoint passes back. Encore maps it to a `404`, returns the message `sql: no rows in result set`, and records the error in the trace. Encore captures the same trace data in every environment. In a multi-service app, traces follow requests across services, Pub/Sub messages, and cron jobs. Learn more in the [tracing documentation](https://encore.dev/docs/tracing). ### Inspect the database Now look at the data written by the query. Open **DB Explorer** in the local dashboard and select the `url` table. You'll see one row for each shortened URL, and from here you can browse and edit rows, add records, and run ad hoc SQL in the **SQL console**. ![Browsing the url table in the DB Explorer](https://encore.dev/assets/docs/qs_db_explorer.png) ### How Encore provisions your database You didn't create the database or run its migrations yourself. The `SQLDatabase` declaration tells Encore to do both, locally and in every cloud environment: - **Locally**, Encore runs PostgreSQL in Docker. - **In the cloud**, deploying to your own account provisions [Amazon RDS](https://encore.dev/docs/platform/infrastructure/aws#databases) on AWS or [Cloud SQL](https://encore.dev/docs/platform/infrastructure/gcp#databases) on GCP, with [Neon](https://encore.dev/docs/platform/infrastructure/neon) supported on both. There's no Terraform to write and no connection strings or credentials to manage, and the [Cloud Dashboard](https://app.encore.dev) gives you this same database explorer for your cloud environments. ## 5. Deploy your app [Encore's cloud platform](https://encore.dev/docs/platform/introduction) includes free development hosting. For production, it can deploy to your own AWS or GCP account. ### Prefer to self-host? Build a Docker image for your app: ```shell $ encore build docker --config infra-config.json MY-IMAGE:TAG ``` This compiles your app and produces an image you can deploy anywhere. The [infra config file](https://encore.dev/docs/self-host/configure-infra) tells the app how to connect to your database. See the [self-host docs](https://encore.dev/docs/self-host/build). If you skipped account creation in step 1, first [link your local app to Encore](https://encore.dev/docs/platform/deploy/deploying#already-have-a-local-app). `encore app create` set up a git repository with an `encore` remote. Commit your changes and push to deploy: ```shell $ git add -A . $ git commit -m 'Deploy my app' $ git push encore remote: main: triggered deploy https://app.encore.dev//envs/staging/deploys/... ``` Encore's cloud platform builds and tests your app, provisions the PostgreSQL database, and deploys everything to `staging`, a free environment on Encore Cloud. Open the URL to follow the deployment in the Cloud Dashboard. Once the deployment finishes, call the API using its staging URL: ```shell $ curl -X POST https://staging-.encr.app/url -d '{"url":"https://encore.dev"}' ``` Then inspect the trace in the Cloud Dashboard. The tracing works just as it did locally, but the app now runs against cloud infrastructure. ## 6. Take a look at the code **Encore.ts:** Open `url/url.ts`. This is the endpoint you just traced: ```ts -- url/url.ts -- import { api } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { randomBytes } from "node:crypto"; // 'url' database is used to store the URLs that are being shortened. const db = new SQLDatabase("url", { migrations: "./migrations" }); // shorten shortens a URL. export const shorten = api( { expose: true, auth: false, method: "POST", path: "/url" }, async ({ url }: ShortenParams): Promise => { const id = randomBytes(6).toString("base64url"); await db.exec` INSERT INTO url (id, original_url) VALUES (${id}, ${url}) `; return { id, url }; } ); ``` This is standard TypeScript. Two declarations connect it to Encore: - `api` defines a public endpoint. Encore handles routing, request validation, error handling, observability, and API documentation. - `new SQLDatabase(...)` declares the database. Encore uses the declaration to provision PostgreSQL and include its queries in traces. The `shorten` endpoint belongs to the `url` service, defined by `encore.service.ts` in the same folder: ```ts -- url/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("url"); ``` Everything under `url/` belongs to the `url` service. To add another service, create a directory with an `encore.service.ts` that exports a new `Service`. Learn more about [services](https://encore.dev/docs/ts/primitives/services), [APIs](https://encore.dev/docs/ts/primitives/defining-apis), and [databases](https://encore.dev/docs/ts/primitives/databases). **Encore.go:** Open `url/url.go`. This is the endpoint you just traced: ```go -- url/url.go -- package url import ( "context" "encore.dev/storage/sqldb" ) // Shorten shortens a URL. // //encore:api public method=POST path=/url func Shorten(ctx context.Context, p *ShortenParams) (*URL, error) { id, err := generateID() if err != nil { return nil, err } else if err := insert(ctx, id, p.URL); err != nil { return nil, err } return &URL{ID: id, URL: p.URL}, nil } // 'url' database is used to store the URLs that are being shortened. var db = sqldb.NewDatabase("url", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` This is standard Go. Two declarations connect it to Encore: - `//encore:api public` defines `Shorten` as a public endpoint in the `url` service. Encore handles routing, request validation, error handling, observability, and API documentation. - `sqldb.NewDatabase(...)` declares the database. Encore uses the declaration to provision PostgreSQL and include its queries in traces. To add another service, create a Go package and annotate its endpoints the same way. Learn more about [defining APIs](https://encore.dev/docs/go/primitives/defining-apis) and [databases](https://encore.dev/docs/go/primitives/databases). ## 7. Add some infrastructure **Encore.ts:** The [Encore.ts SDK](https://encore.dev/docs/ts) lets you declare infrastructure directly in code, including Pub/Sub topics, object storage, and cron jobs. **Encore.go:** The [Encore.go SDK](https://encore.dev/docs/go) lets you declare infrastructure directly in code, including Pub/Sub topics, object storage, and cron jobs. The next three steps build on each other: add a second service, give it its own database, then connect the two with Pub/Sub. Each step takes only a few lines of code. There's nothing else to install, configure, or wire up. Save your changes and Encore provisions the infrastructure and reloads your app. ### Add a service Splitting a backend into services usually means wiring up service discovery, clients, and inter-service authentication. With Encore, you create a directory and call the other service like a function. Tracing follows the request across both services automatically. You'll add a `stats` service that reports how many URLs have been shortened, by calling the `url` service to count them. **Encore.ts:** A directory containing an `encore.service.ts` defines a service, so add `stats/` next to the existing `url/` directory: ``` /your-app-name ├── url // existing url service │ ├── encore.service.ts │ ├── migrations │ └── url.ts └── stats // new stats service ├── encore.service.ts └── stats.ts ``` Create `stats/encore.service.ts`: ```ts -- stats/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("stats"); ``` Next, create `stats/stats.ts`. Importing from `~encore/clients` lets you call the `url` service like a local function: ```ts -- stats/stats.ts -- import { api } from "encore.dev/api"; import { url } from "~encore/clients"; export const count = api( { expose: true, method: "GET", path: "/stats" }, async (): Promise<{ total: number }> => { const { urls } = await url.list(); return { total: urls.length }; } ); ``` **Encore.go:** A Go package with an API endpoint defines a service, so add `stats/` next to the existing `url/` directory: ``` /your-app-name ├── url // existing url service │ ├── migrations │ ├── index.go │ └── url.go └── stats // new stats service └── stats.go ``` The `url` service doesn't have an endpoint that returns a total, so add one to `url/url.go`. Marking it `private` makes it callable from other services, but not from outside the application: ```go HL url/url.go 4:15 -- url/url.go -- var db = sqldb.NewDatabase("url", sqldb.DatabaseConfig{ Migrations: "./migrations", }) type CountResponse struct { Total int } // The path can't be /url/count, which would conflict with /url/:id. // //encore:api private method=GET path=/url-count func Count(ctx context.Context) (*CountResponse, error) { var total int err := db.QueryRow(ctx, `SELECT count(*) FROM url`).Scan(&total) return &CountResponse{Total: total}, err } ``` Create `stats/stats.go`. Importing `encore.app/url` lets you call the `url` service like a local function: ```go -- stats/stats.go -- package stats import ( "context" "encore.app/url" ) type Response struct { Total int } //encore:api public method=GET path=/stats func Count(ctx context.Context) (*Response, error) { resp, err := url.Count(ctx) if err != nil { return nil, err } return &Response{Total: resp.Total}, nil } ``` **Encore.ts:** In the **API Explorer**, call `stats.count`, then open its trace. You'll see a single request spanning **two services**, including the call to `url.list` and its database query. **Encore.go:** In the **API Explorer**, call `stats.Count`, then open its trace. You'll see a single request spanning **two services**, including the call to `url.Count` and its database query. Open **Architecture** in the local dashboard to see the new dependency between `stats` and `url`. ![The stats service calling url on the Architecture page](https://encore.dev/assets/docs/qs_step6_services.png) ### Add a database Now give `stats` its own database. Databases are declared in code the same way services are, with no cluster to provision or connection string to manage, and queries to every database appear in your traces. You'll give `stats` a table to track how many times each URL has been shortened. For now you'll write to it from an endpoint you call by hand; in the next step, Pub/Sub will fill it in automatically. **Encore.ts:** Create the migration that defines its schema: ```sql -- stats/migrations/1_create_clicks.up.sql -- CREATE TABLE clicks ( id TEXT PRIMARY KEY, clicks BIGINT NOT NULL DEFAULT 0 ); ``` Declare the database and an endpoint to record clicks in `stats/stats.ts`: ```ts HL stats/stats.ts 2:15 -- stats/stats.ts -- import { api } from "encore.dev/api"; import { url } from "~encore/clients"; import { SQLDatabase } from "encore.dev/storage/sqldb"; export const db = new SQLDatabase("stats", { migrations: "./migrations" }); export const record = api( { expose: true, method: "POST", path: "/stats/:id" }, async ({ id }: { id: string }): Promise => { await db.exec` INSERT INTO clicks (id, clicks) VALUES (${id}, 1) ON CONFLICT (id) DO UPDATE SET clicks = clicks.clicks + 1 `; } ); export const count = api( { expose: true, method: "GET", path: "/stats" }, async (): Promise<{ total: number }> => { const { urls } = await url.list(); return { total: urls.length }; } ); ``` **Encore.go:** Create the migration that defines its schema: ```sql -- stats/migrations/1_create_clicks.up.sql -- CREATE TABLE clicks ( id TEXT PRIMARY KEY, clicks BIGINT NOT NULL DEFAULT 0 ); ``` Declare the database and an endpoint to record clicks in `stats/stats.go`: ```go HL stats/stats.go 6:20 -- stats/stats.go -- package stats import ( "context" "encore.app/url" "encore.dev/storage/sqldb" ) var db = sqldb.NewDatabase("stats", sqldb.DatabaseConfig{ Migrations: "./migrations", }) //encore:api public method=POST path=/stats/:id func Record(ctx context.Context, id string) error { _, err := db.Exec(ctx, ` INSERT INTO clicks (id, clicks) VALUES ($1, 1) ON CONFLICT (id) DO UPDATE SET clicks = clicks.clicks + 1 `, id) return err } type Response struct { Total int } //encore:api public method=GET path=/stats func Count(ctx context.Context) (*Response, error) { resp, err := url.Count(ctx) if err != nil { return nil, err } return &Response{Total: resp.Total}, nil } ``` Encore provisions new databases at startup, not on hot reload, so restart the app: ```shell $ encore run ``` **Encore.ts:** In the **API Explorer**, call `stats.record` with any id, a few times over. Open **DB Explorer**, switch to the `stats` database, and select the `clicks` table to see the count increase. **Encore.go:** In the **API Explorer**, call `stats.Record` with any id, a few times over. Open **DB Explorer**, switch to the `stats` database, and select the `clicks` table to see the count increase. ![The clicks table in the stats database, shown in the DB Explorer](https://encore.dev/assets/docs/qs_step6_database.png) ### Add Pub/Sub Move work out of the request path so slower tasks don't block the response. Declare a Pub/Sub topic in code and Encore runs it locally. When you deploy, Encore provisions GCP Pub/Sub or Amazon SNS/SQS, with no broker to configure or credentials to connect. You'll publish an event whenever a URL is shortened, and have `stats` subscribe to it. The `url` service no longer needs to know that `stats` exists, and the counting happens outside the request. **Encore.ts:** First, add the Pub/Sub import to `url/url.ts`: ```ts HL url/url.ts 0:0 -- url/url.ts -- import { Topic } from "encore.dev/pubsub"; ``` Then declare the topic alongside the existing database: ```ts HL url/url.ts 2:9 -- url/url.ts -- const db = new SQLDatabase("url", { migrations: "./migrations" }); export interface ShortenedEvent { id: string; url: string; } export const shortened = new Topic("shortened", { deliveryGuarantee: "at-least-once", }); ``` Then publish an event from `shorten`, immediately after the `INSERT`: ```ts HL url/url.ts 8:8 -- url/url.ts -- export const shorten = api( { expose: true, auth: false, method: "POST", path: "/url" }, async ({ url }: ShortenParams): Promise => { const id = randomBytes(6).toString("base64url"); await db.exec` INSERT INTO url (id, original_url) VALUES (${id}, ${url}) `; await shortened.publish({ id, url }); return { id, url }; } ); ``` Finally, create `stats/subscriptions.ts` to subscribe from the `stats` service: ```ts -- stats/subscriptions.ts -- import { Subscription } from "encore.dev/pubsub"; import { shortened } from "../url/url"; import { db } from "./stats"; const _ = new Subscription(shortened, "count-shortened", { handler: async (event) => { await db.exec` INSERT INTO clicks (id, clicks) VALUES (${event.id}, 1) ON CONFLICT (id) DO UPDATE SET clicks = clicks.clicks + 1 `; }, }); ``` **Encore.go:** First, add the Pub/Sub import to `url/url.go`: ```go HL url/url.go 0:0 -- url/url.go -- import "encore.dev/pubsub" ``` Then declare the topic alongside the existing database: ```go HL url/url.go 4:11 -- url/url.go -- var db = sqldb.NewDatabase("url", sqldb.DatabaseConfig{ Migrations: "./migrations", }) type ShortenedEvent struct { ID string URL string } var Shortened = pubsub.NewTopic[*ShortenedEvent]("shortened", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) ``` Then publish an event from `Shorten`, immediately after the insert: ```go HL url/url.go 8:10 -- url/url.go -- //encore:api public method=POST path=/url func Shorten(ctx context.Context, p *ShortenParams) (*URL, error) { id, err := generateID() if err != nil { return nil, err } else if err := insert(ctx, id, p.URL); err != nil { return nil, err } if _, err := Shortened.Publish(ctx, &ShortenedEvent{ID: id, URL: p.URL}); err != nil { return nil, err } return &URL{ID: id, URL: p.URL}, nil } ``` Finally, create `stats/subscriptions.go` to subscribe from the `stats` service: ```go -- stats/subscriptions.go -- package stats import ( "context" "encore.app/url" "encore.dev/pubsub" ) var _ = pubsub.NewSubscription( url.Shortened, "count-shortened", pubsub.SubscriptionConfig[*url.ShortenedEvent]{ Handler: func(ctx context.Context, event *url.ShortenedEvent) error { _, err := db.Exec(ctx, ` INSERT INTO clicks (id, clicks) VALUES ($1, 1) ON CONFLICT (id) DO UPDATE SET clicks = clicks.clicks + 1 `, event.ID) return err }, }, ) ``` In the **API Explorer**, call `url.shorten` a few times, then open one of the traces. The publish appears in the request trace, and the subscription runs in a separate trace linked to it. ![A shorten request trace showing the database query and the published event](https://encore.dev/assets/docs/qs_step6_pubsub.png) Check the `clicks` table in the **DB Explorer** again. The rows are now written by the subscriber, without `url.shorten` waiting for it. Open **Architecture** in the local dashboard to see the topic and subscription alongside your services. ![The shortened topic alongside the url and stats services on the Architecture page](https://encore.dev/assets/docs/qs_step6_pubsub_flow.png) ## 8. Deploy your changes Commit your changes and push them: ```shell $ git add -A . $ git commit -m 'Add stats service' $ git push encore ``` Encore sees the new `stats` database and `shortened` topic in your code, provisions them in `staging`, and then deploys the new version. Every push to your main branch deploys the same way. Once the deployment finishes, shorten a URL and check the stats at your staging URL: ```shell $ curl -X POST https://staging-.encr.app/url -d '{"url":"https://encore.dev"}' $ curl https://staging-.encr.app/stats ``` From the Cloud Dashboard you can also [connect your cloud account](https://encore.dev/docs/platform/deploy/own-cloud) and [integrate with GitHub](https://encore.dev/docs/platform/integrations/github). If a deploy fails, see [Fixing a failed deploy](https://encore.dev/docs/platform/deploy/troubleshooting). ## What's next? - Explore the **Encore.ts:** [Encore.ts primitives](https://encore.dev/docs/ts/primitives)**Encore.go:** [Encore.go primitives](https://encore.dev/docs/go/primitives) to declare Pub/Sub, object storage, cron jobs, secrets, and more. - Build with your AI coding tool. With Encore's AI instructions installed, your agent knows how to declare infrastructure in code, so a prompt like _"Add a cron job that deletes URLs older than 30 days and logs how many it removed"_ is enough. See [AI tools integration](https://encore.dev/docs/ai-integration). - Follow the [Uptime Monitor tutorial](https://encore.dev/docs/tutorials/uptime) to build a multi-service, event-driven system. - Join the Encore community on [Discord](https://encore.dev/discord) to ask questions and meet other developers. --- Source: https://encore.dev/docs/understanding-encore Section: Getting started # Understanding Encore > How application code becomes running infrastructure Encore is a platform that lets engineering teams define backend infrastructure through type-safe primitives in application code, so developers (and their agents) can work with application resources like services, APIs, databases, queues, and caches, while platform teams keep control over how those resources are configured and run. Infrastructure that would normally be described in Terraform can instead be derived from the application, with Encore using static analysis to work out service dependencies, IAM, and how resources connect, while environment-specific parts like sizing, scaling, and regions stay under the platform team's control, whether that configuration lives in Encore or continues to be managed through [existing Terraform](https://encore.dev/docs/platform/migration/from-terraform) during a gradual adoption. The link between these capabilities is the **Encore Application Model**: a description of your application's services, APIs, resources, and the relationships between them. Encore derives the model from static analysis of your source code, without running it. Encore uses these application definitions across local development, preview environments, and production, so locally it can start and wire the infrastructure the application needs, while pull requests can run in isolated preview environments where developers and agents can test changes end-to-end against an environment that closely matches production. For production, Encore combines those application definitions with the [platform configuration](https://encore.dev/docs/platform/introduction) to provision the required infrastructure in your own AWS or GCP account, with logs, metrics, tracing, and observability integrated into the workflow. Encore runs a declared SQL database as a Postgres container locally, and the platform provisions it as RDS or Cloud SQL in the cloud. A declared Pub/Sub topic runs on a local broker, then on SNS and SQS or Google Cloud Pub/Sub, and so on. ## The core idea Consider this declaration of a SQL database, one of Encore's infrastructure primitives for [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go): **Encore.ts:** ```ts const db = new SQLDatabase("orders", { migrations: "./migrations" }); ``` **Encore.go:** ```go var db = sqldb.NewDatabase("orders", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` From this declaration, the parser records the database name and its migration directory. As it analyzes the rest of the application, it also records which services query the database. Running `encore run` creates a Postgres database in a local container. Deploying with [Encore's cloud platform](https://encore.dev/docs/platform/introduction) provisions a managed Postgres instance in your own cloud account and applies the same migrations. See [Development workflow](https://encore.dev/docs/platform/workflow) for using one declaration across local, preview, and production environments. Encore rebuilds the model from source on every `encore run` and every build, keeping it synchronized with the code it describes. During the build, the parser validates the resource declarations it discovers and reports invalid declarations. The SDK, parser and compiler that produce the model are all [Open Source](https://github.com/encoredev/encore). The development dashboard renders the model as an architecture diagram, where each box represents a service and shows its public, authenticated, and private endpoints and the databases it uses. Hovering over a service highlights its callers, API calls, and resource dependencies: ## What the model enables Because Encore understands the application rather than only its deployment configuration, the same model supports the whole development workflow. Two separate pieces read it. ### What the open source tooling does The SDKs, parser, compiler, CLI, and runtime are [open source](https://github.com/encoredev/encore) and work on their own: - **Local infrastructure:** `encore run` starts local implementations of declared resources without a separate Docker Compose or emulator configuration. - **Build-time validation:** the compiler reports invalid declarations and resource usage before deployment. - **Development tooling:** the local dashboard provides API documentation, architecture diagrams, logs, traces, and a database explorer. - **Type-safe communication:** Encore validates API schemas and generates clients for calling services and frontends. If you stop here, you can [build a Docker image](https://encore.dev/docs/self-host/build) from the same model and [deploy it yourself](https://encore.dev/docs/self-host/configure-infra). ### What Encore's cloud platform does [Encore's cloud platform](https://encore.dev/docs/platform/introduction) is optional. It reads the same model to provision and operate infrastructure in **your own cloud account**: - **[Provisioning infrastructure](https://encore.dev/docs/platform/infrastructure/infra)** in [your own AWS or GCP account](https://encore.dev/docs/platform/deploy/own-cloud), with [per-environment](https://encore.dev/docs/platform/deploy/environments) capacity, networking, and backup settings you [configure](https://encore.dev/docs/platform/infrastructure/configuration). - **[Deployments](https://encore.dev/docs/platform/deploy/deploying)** from a `git push`, with an isolated [preview environment](https://encore.dev/docs/platform/deploy/preview-environments) per pull request where developers and agents can test changes end-to-end against infrastructure that closely matches production. - **[Least-privilege IAM](https://encore.dev/docs/platform/deploy/security):** resource usage determines which services can call an API or access a database, bucket, or topic. - **[Tracing](https://encore.dev/docs/platform/observability/tracing) and [metrics](https://encore.dev/docs/platform/observability/metrics)** for deployed environments, plus a cloud dashboard for logs, traces, and databases. You own the cloud account and the provisioned infrastructure. Your application does not depend on the platform at runtime, and you can use infrastructure outside the model alongside either approach. ## Inside the model - Services, and the directory or package each one lives in - API endpoints, with the full type schema of every request and response - SQL databases and their migration directories - Pub/Sub topics and subscriptions - Cron jobs, object storage buckets, caches and secrets - Middleware and API gateways Running a build without supplying infrastructure configuration prints the part of the model that has to be satisfied: ``` $ encore build docker myapp:latest Your infra configuration is incomplete Missing Resource Configurations: Secrets : SlackWebhookURL Databases : monitor, site Subscriptions: uptime-transition/slack-notification, site.added/check-site Topics : uptime-transition, site.added ``` ## Deriving IAM and configuration The model records how resources are used. Encore uses this information to derive IAM permissions and other configuration. For example, two services may use the same `uploads` bucket: **Encore.ts:** ```ts // in the ingest service await uploads.upload("q3.csv", csvBytes); // in the reports service const csv = await uploads.download("q3.csv"); ``` **Encore.go:** ```go // in the ingest service _, err := uploads.Upload(ctx, "q3.csv").Write(csvBytes) // in the reports service csv, err := uploads.Download(ctx, "q3.csv").All() ``` The parser records the download in `reports` and the upload in `ingest`. Encore grants `reports` read access to the bucket and `ingest` write access. Bucket usage distinguishes operations such as reading object contents, accessing metadata, and generating signed upload URLs. The model also records which services publish to each topic and call each endpoint. **Encore.ts:** The model drives provisioning, [request validation](https://encore.dev/docs/ts/primitives/validation) against the declared schemas, [generated clients](https://encore.dev/docs/ts/cli/client-generation), [API documentation](https://encore.dev/docs/ts/develop/api-docs), and [distributed tracing](https://encore.dev/docs/tracing) across service boundaries. These features use the application model as their shared source of service, resource, and schema information. **Encore.go:** The model drives provisioning, [request validation](https://encore.dev/docs/go/develop/validation) against the declared schemas, [generated clients](https://encore.dev/docs/go/cli/client-generation), [API documentation](https://encore.dev/docs/go/develop/api-docs), and [distributed tracing](https://encore.dev/docs/tracing) across service boundaries. These features use the application model as their shared source of service, resource, and schema information. ## Requirements on your code Resource declarations are read from your source without running it, which constrains how you write them. ### Names must be string literals A name assembled from a variable or a template fails the build: **Encore.ts:** ```ts const regions = ["eu", "us", "ap"]; export const orderTopics = regions.map( (r) => new Topic(`orders-${r}`, { deliveryGuarantee: "at-least-once" }), ); ``` **Encore.go:** ```go regions := []string{"eu", "us", "ap"} for _, r := range regions { pubsub.NewTopic[string](fmt.Sprintf("orders-%s", r), pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) } ``` ``` error: expected string literal --> orders/topics.ts:5:10 | 5 | (r) => new Topic(`orders-${r}`, { deliveryGuarantee: "at-least-once" }), | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ``` ### Declarations must be assigned at module scope The parser collects top-level bindings. Constructor calls inside functions, conditionals, or loops are not included in the application model: **Encore.ts:** ```ts if (process.env.SHARD) { // Not part of the model, so never provisioned. new SQLDatabase("shard", { migrations: "./migrations" }); } ``` **Encore.go:** ```go func init() { // Not part of the model, so never provisioned. sqldb.NewDatabase("shard", sqldb.DatabaseConfig{Migrations: "./migrations"}) } ``` API endpoint declarations must be exported. Service definitions must be the module's default export. ## Infrastructure outside the model **Encore.ts:** The application model supports a fixed set of resource types. Other infrastructure and third-party services are not included in the model. Provision resources such as search clusters or queues determined at runtime separately, connect to them as external dependencies, and store their connection details in a [secret](https://encore.dev/docs/ts/primitives/secrets): **Encore.go:** The application model supports a fixed set of resource types. Other infrastructure and third-party services are not included in the model. Provision resources such as search clusters or queues determined at runtime separately, connect to them as external dependencies, and store their connection details in a [secret](https://encore.dev/docs/go/primitives/secrets): **Encore.ts:** ```ts import { secret } from "encore.dev/config"; const searchEndpoint = secret("SearchEndpoint"); export const search = api( { expose: true, method: "GET", path: "/search" }, async ({ q }: { q: string }): Promise => { const resp = await fetch(`${searchEndpoint()}/query?q=${q}`); return resp.json(); }, ); ``` **Encore.go:** ```go var secrets struct { SearchEndpoint string } //encore:api public method=GET path=/search func Search(ctx context.Context, q string) (*Results, error) { resp, err := http.Get(secrets.SearchEndpoint + "/query?q=" + q) // ... } ``` Encore knows about the secret and the endpoint that uses it, so both appear in the model, while the cluster behind the endpoint does not. The [Terraform Provider](https://encore.dev/docs/platform/integrations/terraform) also exposes data sources for infrastructure provisioned by Encore, allowing existing Terraform configurations to reference an Encore database or topic by name. ## Frequently asked questions ### Can I see the model? Yes. The [local development dashboard](https://encore.dev/docs/dev-dash) renders it as a service catalog and an architecture diagram, so you can inspect the services, endpoints and resources the parser found. ### Does my source code get sent anywhere? Static analysis itself does not send your source code anywhere. The open source parser runs in the environment where the build is performed, including locally during `encore run`. ### What if someone changes a resource in the cloud console? The application model and the state of provisioned infrastructure are managed separately. The model is derived from the application code. If a resource is modified outside Encore, the next deploy detects the change and updates Encore's record instead of overwriting it. See [managing infrastructure](https://encore.dev/docs/platform/infrastructure/managing-infrastructure). ### Does static analysis constrain the rest of my code? No. The requirements above apply to resource declarations. Static analysis does not otherwise restrict the libraries, patterns, or abstractions used in your application logic. ### Does static analysis slow down my builds? No. The parser runs during the build, before infrastructure provisioning, and adds little next to compilation and building the container image. ### Is the model specific to TypeScript? No. Encore builds the same application model for TypeScript and Go applications, using a parser designed for each language. --- Source: https://encore.dev/docs/ai-integration Section: Getting started # AI-assisted development > Give coding agents application context and build-time guardrails A coding agent can make better changes when it understands the system around the file it is editing. Encore gives it that context through the [Encore Application Model](https://encore.dev/docs/understanding-encore), which contains the application's services, APIs, schemas, infrastructure resources, and the relationships between them. The model is derived from declarations made with Encore's open source Infra SDK. APIs and infrastructure resources are type-safe, and Encore validates resource declarations during the build. Invalid or statically unresolved declarations produce build errors. [Deployment-specific infrastructure settings](https://encore.dev/docs/platform/infrastructure/configuration), including capacity, networking, and backups, are configured per environment. They are not part of the resource declarations an agent edits in the application code. Together, these constraints provide guardrails for AI-assisted development: an agent can declare what the application uses, while the type system, parser, and environment configuration limit what can be expressed or changed from the application source. ## How the integrations fit together Encore provides four complementary ways to help coding agents work with an application: | Integration | Purpose | | --- | --- | | [Encore skills](https://github.com/encoredev/skills) | Teach the agent how to set up Encore and how to use it: installing the CLI, creating, running, and deploying apps, and working with APIs, databases, Pub/Sub, auth, testing, and more. | | Project rules | Explain Encore conventions and project-specific coding patterns to the agent. | | [Local MCP server](https://encore.dev/docs/mcp) | Expose the application model, database schemas, APIs, local data, and traces as structured tools. | | [Cloud MCP server](https://encore.dev/docs/platform/cloud-mcp) | Expose deployed environments on Encore Cloud: deploys and build logs, traces, metrics, and infrastructure. | Skills and rules guide how an agent sets up and writes code. The MCP servers let it inspect and interact with the application, locally and in its deployed environments. ## Install Encore skills Start here. Install [Encore's skills](https://github.com/encoredev/skills) in your coding agent: ```bash npx skills add encoredev/skills ``` The command asks which skills to install and for which agents. To install all of them without prompts, name your agent: ```bash # For the current project npx skills add encoredev/skills --skill '*' -a claude-code -y # For all your projects npx skills add encoredev/skills --skill '*' -a claude-code -g -y ``` Replace `claude-code` with your agent, such as `cursor`, `codex`, or `github-copilot`. The [skills CLI](https://github.com/vercel-labs/skills) supports [many more agents](https://github.com/vercel-labs/skills/blob/main/docs/agents.md), and needs Node.js 22.20 or later. To see the skills before installing them, run `npx skills add encoredev/skills --list`. In Claude Code, you can install them as a plugin instead: ```bash claude plugin marketplace add encoredev/skills claude plugin install encore-skills@encore-skills ``` Most agents load new skills when a session starts, so restart your agent after installing them. The skills cover: - **Setup:** installing the CLI, logging in, creating an app, linking it to Encore Cloud, running it locally, and connecting the MCP servers. - **Building:** APIs, services, databases, Pub/Sub, cron jobs, object storage, caching, secrets, auth, webhooks, logging, frontends, and tests, for Encore.ts and Encore.go. - **Shipping:** deploying to Encore Cloud and fixing failed deploys. - **Migrating:** converting an existing backend to Encore, with a plan the agent tracks in the repository. See [Migrate using an AI agent](https://encore.dev/docs/ai-migration). ### Let your agent set up Encore An agent can set up Encore from scratch, including installing the skills. Tell it: ``` Read https://encore.dev/agents/getting-started.md and follow it to create a new Encore app. ``` It installs the skills and the Encore CLI, asks you to log in, creates the app, checks that it runs, connects the MCP servers, and deploys it. To convert an existing backend instead, point it to https://encore.dev/agents/migrate.md. ## Add project rules ### New projects Run `encore app create` and select the AI tool you use. Encore generates the corresponding project rules and, for supported MCP hosts, the MCP configuration. ```bash encore app create ``` ![](https://encore.dev/assets/docs/initllm.png) To skip the prompt, pass the tool with `--llm-rules`, for example `encore app create my-app --example=ts/hello-world --llm-rules=claudecode`. ### Existing projects From the application directory, run: ```bash encore llm-rules init ``` Select your AI tool when prompted, or pass it with `--llm-rules`. The command generates the appropriate rules file, such as `CLAUDE.md` or Cursor rules, and configures the local MCP server for supported hosts. The generated formats support tools and conventions including Cursor, Claude Code, VS Code, `AGENTS.md`, and Zed. ## Connect the local MCP server The Encore CLI includes a local [Model Context Protocol](https://modelcontextprotocol.io/introduction) server. It exposes tools for inspecting the application model, reading database schemas, querying local databases, calling APIs, and reading traces. Configuration is generated automatically for supported hosts when you create a project or run `encore llm-rules init`. Use the following setup when you need to connect a host manually. ### Start the server From the Encore application directory, run: ```bash encore mcp start ``` The command prints the SSE URL and stdio command for the application, including its app ID. Use that app ID in the configurations below: an app that isn't linked to Encore Cloud has no ID in `encore.app`. Keep the server running when connecting over SSE. Hosts configured with the stdio command start their own MCP process. ### Connect Cursor Use the button below after replacing `your-app-id` with the app ID that `encore mcp start` prints: [![Add encore-local MCP server to Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=encore-local&config=eyJjb21tYW5kIjoiZW5jb3JlIG1jcCBydW4gLS1hcHA9eW91ci1hcHAtaWQifQ%3D%3D) To configure it manually, create `.cursor/mcp.json`: ```json { "mcpServers": { "encore-local": { "command": "encore", "args": ["mcp", "run", "--app=your-app-id"] } } } ``` ### Connect Claude Code From the Encore application directory, run: ```bash claude mcp add --transport stdio encore-local -- encore mcp run --app=your-app-id ``` Verify the connection with `claude mcp list`. ## Connect the Cloud MCP server The [Cloud MCP server](https://encore.dev/docs/platform/cloud-mcp) at `https://api.encore.dev/mcp` gives agents your deployed environments on Encore Cloud, including deploys and build logs, production traces, metrics, and infrastructure. In Claude Code: ```bash claude mcp add --transport http encore-cloud https://api.encore.dev/mcp ``` Then run `/mcp`, select `encore-cloud`, and choose **Authenticate** to sign in through your browser. For other tools, and for API keys in CI, see the [Cloud MCP server](https://encore.dev/docs/platform/cloud-mcp) docs. ## What an agent can inspect and test The available actions depend on the tools supported by the agent and the permissions you grant it. Through Encore's local MCP server, an agent can: - Inspect services, APIs, middleware, authentication handlers, and infrastructure resources - Read SQL database schemas and query local application data - Call API endpoints in the running application - Search traces and inspect spans across API calls and Pub/Sub handlers - Inspect Pub/Sub topics, object storage, caches, cron jobs, metrics, and secret metadata - Search and read Encore documentation See the [Local MCP server reference](https://encore.dev/docs/mcp) for the complete tool list. > **Warning** > > MCP tools can read local application data, execute database queries, and call endpoints. Review the MCP configuration, the application it targets, and each tool request according to your AI tool's approval settings. Secret values are not exposed by the secret metadata tool. ## Example workflow Start the application with `encore run`, then ask the agent to make and verify a change: 1. Ask it to add an API, database, topic, or another infrastructure primitive ([TypeScript](https://encore.dev/docs/ts), [Go](https://encore.dev/docs/go)). 2. Build or run the application. Type errors and invalid resource declarations are reported before deployment. 3. Let the agent call the local API and inspect the resulting database state or trace through MCP. 4. Review the code and any environment configuration needed for deployment. For example: - "Add an endpoint that publishes to a Pub/Sub topic, call it, and verify the publish in the trace." - "Inspect the users database schema and add an endpoint that returns accounts created in the last week." - "Create a service with CRUD endpoints backed by PostgreSQL, then run its tests." Local development uses the implementations provided by `encore run`, including PostgreSQL in Docker and local implementations of other declared resources. See [Development workflow](https://encore.dev/docs/platform/workflow) for how the same application declarations are used across local, preview, and production environments. ## Learn more - [Encore skills](https://github.com/encoredev/skills) - [Agent guide: create a new Encore app](https://encore.dev/agents/getting-started.md) - [Understanding Encore](https://encore.dev/docs/understanding-encore) - [Local MCP server reference](https://encore.dev/docs/mcp) - [Cloud MCP server](https://encore.dev/docs/platform/cloud-mcp) - [Migrate using an AI agent](https://encore.dev/docs/ai-migration) --- Source: https://encore.dev/docs/tutorials/uptime Section: Getting started # Building an Uptime Monitor > Learn how to build an event-driven uptime monitoring system In this tutorial you'll build an uptime monitoring system that notifies you in Slack when one of your sites goes down. It uses an event-driven architecture, and takes about 30 minutes. The final result: ![Frontend](https://encore.dev/assets/tutorials/uptime/frontend.png) **Encore.ts:** _[Interactive editor showing the finished `uptimeTS` app — open this page in a browser to see it.]_ **Encore.go:** _[Interactive editor showing the finished `uptime` app — open this page in a browser to see it.]_ ## 1. Create your Encore application > **Note** > > To make it easier to follow along, we've laid out a trail of croissants to guide your way. > Whenever you see a 🥐 it means there's something for you to do. 🥐 Create a new Encore application, using this tutorial project's starting-point branch. This gives you a ready-to-go frontend to use. **Encore.ts:** ```shell $ encore app create uptime --example=github.com/encoredev/example-app-uptime/tree/starting-point-ts ``` **Encore.go:** ```shell $ encore app create uptime --example=github.com/encoredev/example-app-uptime/tree/starting-point ``` First time using Encore? You'll be asked to create a free account, which is needed for secrets and cloud deployments later in this tutorial. **Encore.ts:** The finished backend, in an [automatically generated diagram](https://encore.dev/docs/architecture). White boxes are services, black boxes are Pub/Sub topics: **Encore.go:** The finished backend, in an [automatically generated diagram](https://encore.dev/docs/architecture). White boxes are services, black boxes are Pub/Sub topics: ![Architecture diagram](https://encore.dev/assets/tutorials/uptime/encore-flow.png) ## 2. Create monitor service First, the functionality to check whether a website is up or down. Later we'll store the result in a database so we can detect status changes and send alerts. **Encore.ts:** 🥐 Create a directory named `monitor` containing a file named `encore.service.ts`. ```shell $ mkdir monitor $ touch monitor/encore.service.ts ``` 🥐 Add the following code to `monitor/encore.service.ts`: ```ts -- monitor/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("monitor"); ``` This is how you define services with Encore. Encore will now consider files in the `monitor` directory and all its subdirectories as part of the `monitor` service. 🥐 In the `monitor` directory, create a file named `ping.ts`. 🥐 Add an Encore API endpoint named `ping` that takes a URL as input and returns a response indicating whether the site is up or down. **Encore.go:** 🥐 Create an Encore service named `monitor` containing a file named `ping.go`. ```shell $ mkdir monitor $ touch monitor/ping.go ``` 🥐 Add an Encore API endpoint named `Ping` that takes a URL as input and returns a response indicating whether the site is up or down. **Encore.ts:** ```ts -- monitor/ping.ts -- // Service monitor checks if a website is up or down. import { api } from "encore.dev/api"; export interface PingParams { url: string; } export interface PingResponse { up: boolean; } // Ping pings a specific site and determines whether it's up or down right now. export const ping = api( { expose: true, path: "/ping/:url", method: "GET" }, async ({ url }) => { // If the url does not start with "http:" or "https:", default to "https:". if (!url.startsWith("http:") && !url.startsWith("https:")) { url = "https://" + url; } try { // Make an HTTP request to check if it's up. const resp = await fetch(url, { method: "GET" }); // 2xx and 3xx status codes are considered up const up = resp.status >= 200 && resp.status < 300; return { up }; } catch (err) { return { up: false }; } } ); ``` **Encore.go:** ```go -- monitor/ping.go -- // Service monitor checks if a website is up or down. package monitor import ( "context" "net/http" "strings" ) // PingResponse is the response from the Ping endpoint. type PingResponse struct { Up bool `json:"up"` } // Ping pings a specific site and determines whether it's up or down right now. // //encore:api public path=/ping/*url func Ping(ctx context.Context, url string) (*PingResponse, error) { // If the url does not start with "http:" or "https:", default to "https:". if !strings.HasPrefix(url, "http:") && !strings.HasPrefix(url, "https:") { url = "https://" + url } // Make an HTTP request to check if it's up. req, err := http.NewRequestWithContext(ctx, "GET", url, nil) if err != nil { return nil, err } resp, err := http.DefaultClient.Do(req) if err != nil { return &PingResponse{Up: false}, nil } resp.Body.Close() // 2xx and 3xx status codes are considered up up := resp.StatusCode < 400 return &PingResponse{Up: up}, nil } ``` 🥐 Let's try it! Run `encore run` in your terminal and you should see the service start up. Then open up the Local Development Dashboard at [http://localhost:9400](http://localhost:9400) and try calling the `monitor.ping` endpoint from the API Explorer, passing in `google.com` as the URL. **Encore.ts:** You'll see the response, the logs, and a trace of the request: **Encore.go:** You'll see the response, the logs, and a trace of the request: **Encore.ts:** Or from a terminal, run `curl http://localhost:4000/ping/google.com`. Either way you should see: **Encore.go:** Or from a terminal, run `curl http://localhost:4000/ping/google.com`. Either way you should see: ```json {"up": true} ``` Try `httpstat.us/400` and `some-non-existing-url.com` too; both should respond with `{"up": false}`. ### Add a test **Encore.ts:** 🥐 Add a test so this endpoint doesn't break over time. Create `monitor/ping.test.ts` with the content: **Encore.go:** 🥐 Add a test so this endpoint doesn't break over time. Create `monitor/ping_test.go` with the content: **Encore.ts:** ```ts -- monitor/ping.test.ts -- import { describe, expect, test } from "vitest"; import { ping } from "./ping"; describe("ping", () => { test.each([ // Test both with and without "https://" { site: "google.com", expected: true }, { site: "https://encore.dev", expected: true }, // 4xx and 5xx should considered down. { site: "https://not-a-real-site.xyz", expected: false }, // Invalid URLs should be considered down. { site: "invalid://scheme", expected: false }, ])( `should verify that $site is ${"$expected" ? "up" : "down"}`, async ({ site, expected }) => { const resp = await ping({ url: site }); expect(resp.up).toBe(expected); }, ); }); ``` 🥐 Run `encore test` to check that it all works as expected. You should see something like: ```shell $ encore test DEV v1.3.0 ✓ monitor/ping.test.ts (4) ✓ ping (4) ✓ should verify that 'google.com' is up ✓ should verify that 'https://encore.dev' is up ✓ should verify that 'https://not-a-real-site.xyz' is down ✓ should verify that 'invalid://scheme' is down Test Files 1 passed (1) Tests 4 passed (4) Start at 12:31:03 Duration 460ms (transform 43ms, setup 0ms, collect 59ms, tests 272ms, environment 0ms, prepare 47ms) PASS Waiting for file changes... ``` **Encore.go:** ```go -- monitor/ping_test.go -- package monitor import ( "context" "testing" ) func TestPing(t *testing.T) { ctx := context.Background() tests := []struct { URL string Up bool }{ {"encore.dev", true}, {"google.com", true}, // Test both with and without "https://" {"httpbin.org/status/200", true}, {"https://httpbin.org/status/200", true}, // 4xx and 5xx should considered down. {"httpbin.org/status/400", false}, {"https://httpbin.org/status/500", false}, // Invalid URLs should be considered down. {"invalid://scheme", false}, } for _, test := range tests { resp, err := Ping(ctx, test.URL) if err != nil { t.Errorf("url %s: unexpected error: %v", test.URL, err) } else if resp.Up != test.Up { t.Errorf("url %s: got up=%v, want %v", test.URL, resp.Up, test.Up) } } } ``` 🥐 Run `encore test ./...` to check that it all works as expected. You should see something like: ```shell $ encore test ./... 9:38AM INF starting request endpoint=Ping service=monitor test=TestPing 9:38AM INF request completed code=ok duration=71.861792 endpoint=Ping http_code=200 service=monitor test=TestPing [... lots more lines ...] PASS ok encore.app/monitor 1.660 ``` And if you open the local development dashboard at [localhost:9400](http://localhost:9400), you can also see traces for the tests. ## 3. Create site service Next, keep track of the websites to monitor. **Encore.ts:** Since most of these APIs will be simple "CRUD" (Create/Read/Update/Delete) endpoints, let's build this service using [Knex.js](https://knexjs.org/), an ORM library that makes building CRUD endpoints really simple. 🥐 Create a new service named `site`: ```shell $ mkdir site # Create a new directory in the application root $ touch site/encore.service.ts ``` ```ts -- site/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("site"); ``` 🥐 Add a SQL database to the `site` service by creating a `migrations` directory inside `site`: ```shell $ mkdir site/migrations ``` **Encore.go:** Since most of these APIs will be simple "CRUD" (Create/Read/Update/Delete) endpoints, let's build this service using [GORM](https://gorm.io/), an ORM library that makes building CRUD endpoints really simple. 🥐 Create a new service named `site` with a SQL database: a `site` directory in the application root, with a `migrations` folder inside it: ```shell $ mkdir site $ mkdir site/migrations ``` 🥐 Add a database migration file inside that folder, named `1_create_tables.up.sql`. The file name is important (it must look something like `1_.up.sql`). Add the following contents: **Encore.ts:** ```sql -- site/migrations/1_create_tables.up.sql -- CREATE TABLE site ( id SERIAL PRIMARY KEY, url TEXT NOT NULL UNIQUE ); ``` 🥐 Next, install the Knex.js library and PostgreSQL client: ```shell $ npm i knex pg ``` Now the `site` service itself, with the CRUD endpoints. 🥐 Create `site/site.ts` with the contents: ```ts -- site/site.ts -- import { api } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import knex from "knex"; // Site describes a monitored site. export interface Site { id: number; // ID is a unique ID for the site. url: string; // URL is the site's URL. } // AddParams are the parameters for adding a site to be monitored. export interface AddParams { // URL is the URL of the site. If it doesn't contain a scheme // (like "http:" or "https:") it defaults to "https:". url: string; } // Add a new site to the list of monitored websites. export const add = api( { expose: true, method: "POST", path: "/site" }, async (params: AddParams): Promise => { const site = (await Sites().insert({ url: params.url }, "*"))[0]; return site; }, ); // Get a site by id. export const get = api( { expose: true, method: "GET", path: "/site/:id", auth: false }, async ({ id }: { id: number }): Promise => { const site = await Sites().where("id", id).first(); return site ?? Promise.reject(new Error("site not found")); }, ); // Delete a site by id. export const del = api( { expose: true, method: "DELETE", path: "/site/:id" }, async ({ id }: { id: number }): Promise => { await Sites().where("id", id).delete(); }, ); export interface ListResponse { sites: Site[]; // Sites is the list of monitored sites } // Lists the monitored websites. export const list = api( { expose: true, method: "GET", path: "/site" }, async (): Promise => { const sites = await Sites().select(); return { sites }; }, ); // Define a database named 'site', using the database migrations // in the "./migrations" folder. Encore automatically provisions, // migrates, and connects to the database. const SiteDB = new SQLDatabase("site", { migrations: "./migrations", }); const orm = knex({ client: "pg", connection: SiteDB.connectionString, }); const Sites = () => orm("site"); ``` **Encore.go:** ```sql -- site/migrations/1_create_tables.up.sql -- CREATE TABLE sites ( id BIGSERIAL PRIMARY KEY, url TEXT NOT NULL ); ``` 🥐 Next, install the GORM library and PostgreSQL driver: ```shell $ go get -u gorm.io/gorm gorm.io/driver/postgres ``` Now let's create the `site` service itself. To do this we'll use Encore's support for [dependency injection](https://encore.dev/docs/go/how-to/dependency-injection) to inject the GORM database connection. 🥐 Create `site/service.go` with the contents: ```go -- site/service.go -- // Service site keeps track of which sites to monitor. package site import ( "encore.dev/storage/sqldb" "gorm.io/driver/postgres" "gorm.io/gorm" ) //encore:service type Service struct { db *gorm.DB } // Define a database named 'site', using the database migrations // in the "./migrations" folder. Encore automatically provisions, // migrates, and connects to the database. var db = sqldb.NewDatabase("site", sqldb.DatabaseConfig{ Migrations: "./migrations", }) // initService initializes the site service. // It is automatically called by Encore on service startup. func initService() (*Service, error) { db, err := gorm.Open(postgres.New(postgres.Config{ Conn: db.Stdlib(), })) if err != nil { return nil, err } return &Service{db: db}, nil } ``` 🥐 Create the CRUD endpoints: ```go -- site/get.go -- package site import "context" // Site describes a monitored site. type Site struct { // ID is a unique ID for the site. ID int `json:"id"` // URL is the site's URL. URL string `json:"url"` } // Get gets a site by id. // //encore:api public method=GET path=/site/:siteID func (s *Service) Get(ctx context.Context, siteID int) (*Site, error) { var site Site if err := s.db.Where("id = $1", siteID).First(&site).Error; err != nil { return nil, err } return &site, nil } -- site/add.go -- package site import "context" // AddParams are the parameters for adding a site to be monitored. type AddParams struct { // URL is the URL of the site. If it doesn't contain a scheme // (like "http:" or "https:") it defaults to "https:". URL string `json:"url"` } // Add adds a new site to the list of monitored websites. // //encore:api public method=POST path=/site func (s *Service) Add(ctx context.Context, p *AddParams) (*Site, error) { site := &Site{URL: p.URL} if err := s.db.Create(site).Error; err != nil { return nil, err } return site, nil } -- site/list.go -- package site import "context" type ListResponse struct { // Sites is the list of monitored sites. Sites []*Site `json:"sites"` } // List lists the monitored websites. // //encore:api public method=GET path=/site func (s *Service) List(ctx context.Context) (*ListResponse, error) { var sites []*Site if err := s.db.Find(&sites).Error; err != nil { return nil, err } return &ListResponse{Sites: sites}, nil } -- site/delete.go -- package site import "context" // Delete deletes a site by id. // //encore:api public method=DELETE path=/site/:siteID func (s *Service) Delete(ctx context.Context, siteID int) error { return s.db.Delete(&Site{ID: siteID}).Error } ``` 🥐 Now make sure you have [Docker](https://docker.com) installed and running, and then restart `encore run` to cause the `site` database to be created by Encore. **Encore.ts:** Check the **Infra** tab in the local development dashboard at [localhost:9400](http://localhost:9400) to confirm the database was created, then call `site.add` from the Service Catalog. You can also call the `site.add` endpoint from the terminal: **Encore.go:** Check the **Infra** tab in the local development dashboard at [localhost:9400](http://localhost:9400) to confirm the database was created, then call `site.Add` from the Service Catalog. Or you can call `site.Add` from the terminal: ```shell $ curl -X POST 'http://localhost:4000/site' -d '{"url": "https://encore.dev"}' { "id": 1, "url": "https://encore.dev" } ``` ## 4. Record uptime checks To notify when a site goes down or comes back up, we need its previous state. 🥐 Add a database to the `monitor` service too. Create `monitor/migrations/1_create_tables.up.sql`: ```sql -- monitor/migrations/1_create_tables.up.sql -- CREATE TABLE checks ( id BIGSERIAL PRIMARY KEY, site_id BIGINT NOT NULL, up BOOLEAN NOT NULL, checked_at TIMESTAMP WITH TIME ZONE NOT NULL ); ``` We'll insert a database row every time we check if a site is up. **Encore.ts:** 🥐 Add a new endpoint `check` to the `monitor` service that takes a Site ID, pings the site, and inserts a row in the `checks` table. **Encore.go:** 🥐 Add a new endpoint `Check` to the `monitor` service that takes a Site ID, pings the site, and inserts a row in the `checks` table. **Encore.ts:** For this service we'll use Encore's [`SQLDatabase` class](https://encore.dev/docs/ts/primitives/databases#querying-data) instead of Knex (in order to showcase both approaches). Add the following to `monitor/check.ts`: **Encore.go:** For this service we'll use Encore's [`sqldb` package](https://encore.dev/docs/go/primitives/databases#querying-databases) instead of GORM (in order to showcase both approaches). ```go -- monitor/check.go -- package monitor import ( "context" "encore.app/site" "encore.dev/storage/sqldb" ) // Check checks a single site. // //encore:api public method=POST path=/check/:siteID func Check(ctx context.Context, siteID int) error { site, err := site.Get(ctx, siteID) if err != nil { return err } result, err := Ping(ctx, site.URL) if err != nil { return err } _, err = db.Exec(ctx, ` INSERT INTO checks (site_id, up, checked_at) VALUES ($1, $2, NOW()) `, site.ID, result.Up) return err } // Define a database named 'monitor', using the database migrations // in the "./migrations" folder. Encore automatically provisions, // migrates, and connects to the database. var db = sqldb.NewDatabase("monitor", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` **Encore.ts:** ```ts -- monitor/check.ts -- import { api } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { ping } from "./ping"; import { site } from "~encore/clients"; // Check checks a single site. export const check = api( { expose: true, method: "POST", path: "/check/:siteID" }, async (p: { siteID: number }): Promise<{ up: boolean }> => { const s = await site.get({ id: p.siteID }); const { up } = await ping({ url: s.url }); await MonitorDB.exec` INSERT INTO checks (site_id, up, checked_at) VALUES (${s.id}, ${up}, NOW()) `; return { up }; }, ); // Define a database named 'monitor', using the database migrations // in the "./migrations" folder. Encore automatically provisions, // migrates, and connects to the database. export const MonitorDB = new SQLDatabase("monitor", { migrations: "./migrations", }); ``` 🥐 Restart `encore run` to cause the `monitor` database to be created. The **Infra** tab again confirms the database, and the architecture diagram now also shows the new dependency between the `monitor` and `site` services. **Encore.ts:** We can then call the `monitor.check` endpoint using the id `1` that we got in the last step, and view the trace where we see the database interactions. **Encore.go:** We can then call the `monitor.Check` endpoint using the id `1` that we got in the last step, and view the trace where we see the database interactions. It will look something like this: **Encore.ts:** You can inspect the data in the database explorer, from the **Infra** tab in the local development dashboard, or with `encore db shell `: **Encore.go:** 🥐 Check the data in the database explorer, from the **Infra** tab in the local development dashboard, or with `encore db shell `: ```shell $ encore db shell monitor psql (14.4, server 14.2) Type "help" for help. monitor=> SELECT * FROM checks; id | site_id | up | checked_at ----+---------+----+------------------------------- 1 | 1 | t | 2022-10-21 09:58:30.674265+00 ``` **Encore.ts:** That's everything working. **Encore.go:** That's everything working. ### Add a cron job to check all sites Now check all the tracked sites regularly, so we can respond when one goes down. **Encore.ts:** We'll create a new `checkAll` API endpoint in the `monitor` service that lists all the tracked sites and checks each one. **Encore.go:** We'll create a new `CheckAll` API endpoint in the `monitor` service that lists all the tracked sites and checks each one. **Encore.ts:** 🥐 Extract some of the functionality from the `check` endpoint into a separate function, like so: ```ts -- monitor/check.ts -- import {Site} from "../site/site"; // Check checks a single site. export const check = api( { expose: true, method: "POST", path: "/check/:siteID" }, async (p: { siteID: number }): Promise<{ up: boolean }> => { const s = await site.get({ id: p.siteID }); return doCheck(s); }, ); async function doCheck(site: Site): Promise<{ up: boolean }> { const { up } = await ping({ url: site.url }); await MonitorDB.exec` INSERT INTO checks (site_id, up, checked_at) VALUES (${site.id}, ${up}, NOW()) `; return { up }; } ``` Now the `checkAll` endpoint itself. 🥐 Create the new `checkAll` endpoint inside `monitor/check.ts`: ```ts -- monitor/check.ts -- // CheckAll checks all sites. export const checkAll = api( { expose: true, method: "POST", path: "/check-all" }, async (): Promise => { const sites = await site.list(); await Promise.all(sites.sites.map(doCheck)); }, ); ``` 🥐 Now that we have a `checkAll` endpoint, define a [cron job](https://encore.dev/docs/ts/primitives/cron-jobs) to automatically call it every 1 hour (since this is an example, we don't need to go too crazy and check every minute): ```ts -- monitor/check.ts -- import { CronJob } from "encore.dev/cron"; // Check all tracked sites every 1 hour. const cronJob = new CronJob("check-all", { title: "Check all sites", every: "1h", endpoint: checkAll, }); ``` **Encore.go:** 🥐 Extract some of the functionality from the `Check` endpoint into a separate function, like so: ```go -- monitor/check.go -- // Check checks a single site. // //encore:api public method=POST path=/check/:siteID func Check(ctx context.Context, siteID int) error { site, err := site.Get(ctx, siteID) if err != nil { return err } return check(ctx, site) } func check(ctx context.Context, site *site.Site) error { result, err := Ping(ctx, site.URL) if err != nil { return err } _, err = db.Exec(ctx, ` INSERT INTO checks (site_id, up, checked_at) VALUES ($1, $2, NOW()) `, site.ID, result.Up) return err } ``` Now the `CheckAll` endpoint itself. 🥐 Create the new `CheckAll` endpoint inside `monitor/check.go`: ```go -- monitor/check.go -- import "golang.org/x/sync/errgroup" // CheckAll checks all sites. // //encore:api public method=POST path=/checkall func CheckAll(ctx context.Context) error { // Get all the tracked sites. resp, err := site.List(ctx) if err != nil { return err } // Check up to 8 sites concurrently. g, ctx := errgroup.WithContext(ctx) g.SetLimit(8) for _, site := range resp.Sites { site := site // capture for closure g.Go(func() error { return check(ctx, site) }) } return g.Wait() } ``` This uses [an errgroup](https://pkg.go.dev/golang.org/x/sync/errgroup) to check up to 8 sites concurrently, aborting early if we encounter any error. (Note that a website being down is not treated as an error.) 🥐 Run `go get golang.org/x/sync/errgroup` to install that dependency. 🥐 Now that we have a `CheckAll` endpoint, define a [cron job](https://encore.dev/docs/go/primitives/cron-jobs) to automatically call it every 1 hour (since this is an example, we don't need to go too crazy and check every minute): ```go -- monitor/check.go -- import "encore.dev/cron" // Check all tracked sites every 1 hour. var _ = cron.NewJob("check-all", cron.JobConfig{ Title: "Check all sites", Endpoint: CheckAll, Every: 1 * cron.Hour, }) ``` > **Note** > > **Encore.ts:** > > Cron jobs are not triggered locally, only when the app is deployed to a cloud environment. > > **Encore.go:** > > Cron jobs are not triggered locally, only when the app is deployed to a cloud environment. The frontend needs a way to list all sites with their up/down status. **Encore.ts:** 🥐 Add a file `monitor/status.ts` with the following code: ```ts -- monitor/status.ts -- import { api } from "encore.dev/api"; import { MonitorDB } from "./check"; interface SiteStatus { id: number; up: boolean; checkedAt: string; } // StatusResponse is the response type from the Status endpoint. interface StatusResponse { // Sites contains the current status of all sites, // keyed by the site ID. sites: SiteStatus[]; } // status checks the current up/down status of all monitored sites. export const status = api( { expose: true, path: "/status", method: "GET" }, async (): Promise => { const rows = await MonitorDB.query` SELECT DISTINCT ON (site_id) site_id, up, checked_at FROM checks ORDER BY site_id, checked_at DESC `; const results: SiteStatus[] = []; for await (const row of rows) { results.push({ id: row.site_id, up: row.up, checkedAt: row.checked_at, }); } return { sites: results }; }, ); ``` With the backend working, open [http://localhost:4000/](http://localhost:4000/) to see the frontend. ![Frontend](https://encore.dev/assets/tutorials/uptime/frontend.png) **Encore.go:** 🥐 Add a file in the `monitor` service and name it `status.go`. Add the following code: ```go -- monitor/status.go -- package monitor import ( "context" "time" ) // SiteStatus describes the current status of a site // and when it was last checked. type SiteStatus struct { Up bool `json:"up"` CheckedAt time.Time `json:"checked_at"` } // StatusResponse is the response type from the Status endpoint. type StatusResponse struct { // Sites contains the current status of all sites, // keyed by the site ID. Sites map[int]SiteStatus `json:"sites"` } // Status checks the current up/down status of all monitored sites. // //encore:api public method=GET path=/status func Status(ctx context.Context) (*StatusResponse, error) { rows, err := db.Query(ctx, ` SELECT DISTINCT ON (site_id) site_id, up, checked_at FROM checks ORDER BY site_id, checked_at DESC `) if err != nil { return nil, err } defer rows.Close() result := make(map[int]SiteStatus) for rows.Next() { var siteID int var status SiteStatus if err := rows.Scan(&siteID, &status.Up, &status.CheckedAt); err != nil { return nil, err } result[siteID] = status } if err := rows.Err(); err != nil { return nil, err } return &StatusResponse{Sites: result}, nil } ``` Visit http://localhost:4000/frontend again. You should now see a working frontend listing all sites and their current status. ## 5. Deploy To try your uptime monitor for real, deploy it to the cloud. ### Self-hosting Encore supports building Docker images directly from the CLI, which can then be self-hosted on your own infrastructure of choice. If your app is using infrastructure resources, such as SQL databases, Pub/Sub, or metrics, you will need to supply a [runtime configuration](https://encore.dev/docs/self-host/configure-infra) your Docker image. 🥐 Create a new file `infra-config.json` in the root of your project with the following contents: ```json -- infra-config.json -- { "$schema": "https://encore.dev/schemas/infra.schema.json", "sql_servers": [ { "host": "my-db-host:5432", "databases": { "monitor": { "username": "my-db-owner", "password": {"$env": "DB_PASSWORD"} }, "site": { "username": "my-db-owner", "password": {"$env": "DB_PASSWORD"} } } } ] } ``` The values in this configuration are just examples, you will need to replace them with the correct values for your database. **Encore.ts:** Take a look at our guide for [deploying an Encore app with a PostgreSQL database to Digital Ocean](https://encore.dev/docs/self-host/deploy-digitalocean) for more information. 🥐 Build a Docker image by running `encore build docker --config infra-config.json uptime:v1.0`. This compiles your app on the host machine and produces a Docker image containing it. 🥐 Upload the Docker image to the cloud provider of your choice and run it. Encore's cloud platform handles the infrastructure and DevOps for you, deploying to a free Encore hosted environment (Encore Cloud) or to your own AWS or GCP account. Don't have an Encore account yet? [Link your app first](https://encore.dev/docs/platform/deploy/deploying#already-have-a-local-app). ### Commit changes **Encore.ts:** Encore comes with built-in CI/CD, and the deployment process is as simple as a `git push`. (You can also integrate with GitHub, learn more in the [CI/CD docs](https://encore.dev/docs/platform/deploy/deploying).) 🥐 Deploy your app to a free Encore hosted environment (Encore Cloud) by running: **Encore.go:** Encore comes with built-in CI/CD, and the deployment process is as simple as a `git push`. (You can also integrate with GitHub to activate per Pull Request Preview Environments, learn more in the [CI/CD docs](https://encore.dev/docs/platform/deploy/deploying).) 🥐 Deploy your app to a free Encore hosted environment (Encore Cloud) by running: ```shell $ git add -A . $ git commit -m 'Initial commit' $ git push encore ``` Encore's cloud platform will now build and test your app, provision the needed infrastructure, and deploy it. After triggering the deployment, you will see a URL where you can view its progress in the [Cloud Dashboard](https://app.encore.dev). It will look something like: `https://app.encore.dev/$APP_ID/envs/staging/deploys/...` From the Cloud Dashboard you can also see metrics, trigger Cron Jobs, see traces, and later connect your own AWS or GCP account to use for deployment. **Encore.ts:** 🥐 When the deploy has finished, you can try out your uptime monitor by going to `https://staging-$APP_ID.encr.app`. *Your app is now running in the cloud.* **Encore.go:** 🥐 When the deploy has finished, you can try out your uptime monitor by going to `https://staging-$APP_ID.encr.app/frontend`. *Your Uptime Monitor is now running in the cloud.* ## 6. Publish Pub/Sub events when a site goes down **Encore.ts:** An uptime monitor isn't much use unless it tells you when a site goes down. Let's add a [Pub/Sub topic](https://encore.dev/docs/ts/primitives/pubsub) on which we'll publish a message every time a site transitions from being up to being down, or vice versa. 🥐 Define the topic using Encore's Pub/Sub module in `monitor/check.ts`: ```ts -- monitor/check.ts -- import { Subscription, Topic } from "encore.dev/pubsub"; // TransitionEvent describes a transition of a monitored site // from up->down or from down->up. export interface TransitionEvent { site: Site; // Site is the monitored site in question. up: boolean; // Up specifies whether the site is now up or down (the new value). } // TransitionTopic is a pubsub topic with transition events for when a monitored site // transitions from up->down or from down->up. export const TransitionTopic = new Topic("uptime-transition", { deliveryGuarantee: "at-least-once", }); ``` **Encore.go:** An uptime monitor isn't much use unless it tells you when a site goes down. Let's add a [Pub/Sub topic](https://encore.dev/docs/go/primitives/pubsub) on which we'll publish a message every time a site transitions from being up to being down, or vice versa. 🥐 Define the topic using Encore's Pub/Sub package in a new file, `monitor/alerts.go`: ```go -- monitor/alerts.go -- package monitor import "encore.dev/pubsub" // TransitionEvent describes a transition of a monitored site // from up->down or from down->up. type TransitionEvent struct { // Site is the monitored site in question. Site *site.Site `json:"site"` // Up specifies whether the site is now up or down (the new value). Up bool `json:"up"` } // TransitionTopic is a pubsub topic with transition events for when a monitored site // transitions from up->down or from down->up. var TransitionTopic = pubsub.NewTopic[*TransitionEvent]("uptime-transition", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) ``` Now publish a message on the `TransitionTopic` when a site's up/down state differs from the previous measurement. 🥐 Create a `getPreviousMeasurement` function to report the last up/down state: **Encore.ts:** ```ts -- monitor/check.ts -- // getPreviousMeasurement reports whether the given site was // up or down in the previous measurement. async function getPreviousMeasurement(siteID: number): Promise { const row = await MonitorDB.queryRow` SELECT up FROM checks WHERE site_id = ${siteID} ORDER BY checked_at DESC LIMIT 1 `; return row?.up ?? true; } ``` 🥐 Now add a function to conditionally publish a message if the up/down state differs by modifying the `doCheck` function: ```ts -- monitor/check.ts -- async function doCheck(site: Site): Promise<{ up: boolean }> { const { up } = await ping({ url: site.url }); // Publish a Pub/Sub message if the site transitions // from up->down or from down->up. const wasUp = await getPreviousMeasurement(site.id); if (up !== wasUp) { await TransitionTopic.publish({ site, up }); } await MonitorDB.exec` INSERT INTO checks (site_id, up, checked_at) VALUES (${site.id}, ${up}, NOW()) `; return { up }; } ``` 🥐 Start your app again using `encore run` and open the architecture diagram in the local development dashboard. Now you'll see the Pub/Sub topic as a black box, it should look like this: ![Architecture diagram](https://encore.dev/assets/docs/uptime_tut_flow_2.png) **Encore.go:** ```go -- monitor/alerts.go -- import ( "encore.dev/storage/sqldb" "errors" "context" ) // getPreviousMeasurement reports whether the given site was // up or down in the previous measurement. func getPreviousMeasurement(ctx context.Context, siteID int) (up bool, err error) { err = db.QueryRow(ctx, ` SELECT up FROM checks WHERE site_id = $1 ORDER BY checked_at DESC LIMIT 1 `, siteID).Scan(&up) if errors.Is(err, sqldb.ErrNoRows) { // There was no previous ping; treat this as if the site was up before return true, nil } else if err != nil { return false, err } return up, nil } ``` 🥐 Now add a function to conditionally publish a message if the up/down state differs: ```go -- monitor/alerts.go -- import "encore.app/site" func publishOnTransition(ctx context.Context, site *site.Site, isUp bool) error { wasUp, err := getPreviousMeasurement(ctx, site.ID) if err != nil { return err } if isUp == wasUp { // Nothing to do return nil } _, err = TransitionTopic.Publish(ctx, &TransitionEvent{ Site: site, Up: isUp, }) return err } ``` 🥐 Finally modify the `check` function to call this function: ```go -- monitor/check.go -- func check(ctx context.Context, site *site.Site) error { result, err := Ping(ctx, site.URL) if err != nil { return err } // Publish a Pub/Sub message if the site transitions // from up->down or from down->up. if err := publishOnTransition(ctx, site, result.Up); err != nil { return err } _, err = db.Exec(ctx, ` INSERT INTO checks (site_id, up, checked_at) VALUES ($1, $2, NOW()) `, site.ID, result.Up) return err } ``` The monitoring system now publishes to the `TransitionTopic` whenever a site goes up->down or down->up, without knowing who listens. Next, add a subscriber that posts those events to Slack. ## 7. Send Slack notifications when a site goes down **Encore.ts:** 🥐 Create a new service named `slack`: ```shell $ mkdir slack # Create a new directory in the application root $ touch slack/encore.service.ts ``` ```ts -- slack/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("slack"); ``` 🥐 Add a `slack.ts` file containing the following: ```ts -- slack/slack.ts -- import { api } from "encore.dev/api"; import { secret } from "encore.dev/config"; import log from "encore.dev/log"; export interface NotifyParams { text: string; // the slack message to send } // Sends a Slack message to a pre-configured channel using a // Slack Incoming Webhook (see https://api.slack.com/messaging/webhooks). export const notify = api({}, async ({ text }) => { const url = webhookURL(); if (!url) { log.info("no slack webhook url defined, skipping slack notification"); return; } const resp = await fetch(url, { method: "POST", headers: { 'Content-Type': 'application/json', }, body: JSON.stringify({ content: text }), }); if (resp.status >= 400) { const body = await resp.text(); throw new Error(`slack notification failed: ${resp.status}: ${body}`); } }); // SlackWebhookURL defines the Slack webhook URL to send uptime notifications to. const webhookURL = secret("SlackWebhookURL"); ``` 🥐 Now go to a Slack community of your choice where you have the permission to create a new Incoming Webhook. **Encore.go:** 🥐 Create a Slack service containing the following: ```go -- slack/slack.go -- package slack import ( "bytes" "context" "encoding/json" "fmt" "io" "net/http" ) type NotifyParams struct { // Text is the Slack message text to send. Text string `json:"text"` } // Notify sends a Slack message to a pre-configured channel using a // Slack Incoming Webhook (see https://api.slack.com/messaging/webhooks). // //encore:api private func Notify(ctx context.Context, p *NotifyParams) error { reqBody, err := json.Marshal(p) if err != nil { return err } req, err := http.NewRequestWithContext(ctx, "POST", secrets.SlackWebhookURL, bytes.NewReader(reqBody)) if err != nil { return err } resp, err := http.DefaultClient.Do(req) if err != nil { return err } defer resp.Body.Close() if resp.StatusCode >= 400 { body, _ := io.ReadAll(resp.Body) return fmt.Errorf("notify slack: %s: %s", resp.Status, body) } return nil } var secrets struct { // SlackWebhookURL defines the Slack webhook URL to send // uptime notifications to. SlackWebhookURL string } ``` 🥐 Now go to a Slack community of your choice where you have the permission to create a new Incoming Webhook. 🥐 Once you have the Webhook URL, set it as an Encore secret: ```shell $ encore secret set --type dev,local,pr SlackWebhookURL Enter secret value: ***** Successfully updated development secret SlackWebhookURL. ``` **Encore.ts:** 🥐 Test the `slack.notify` endpoint by calling it via cURL: ```shell $ curl 'http://localhost:4000/slack.notify' -d '{"text": "Testing Slack webhook"}' ``` **Encore.go:** 🥐 Test the `slack.Notify` endpoint by calling it via cURL: ```shell $ curl 'http://localhost:4000/slack.Notify' -d '{"Text": "Testing Slack webhook"}' ``` You should see the *Testing Slack webhook* message appear in the Slack channel you designated for the webhook. 🥐 Now add a Pub/Sub subscriber to notify Slack automatically when a site goes up or down: **Encore.ts:** ```ts -- slack/slack.ts -- import { Subscription } from "encore.dev/pubsub"; import { TransitionTopic } from "../monitor/check"; const _ = new Subscription(TransitionTopic, "slack-notification", { handler: async (event) => { const text = `*${event.site.url} is ${event.up ? "back up." : "down!"}*`; await notify({ text }); }, }); ``` **Encore.go:** ```go -- slack/slack.go -- import ( "encore.dev/pubsub" "encore.app/monitor" ) var _ = pubsub.NewSubscription(monitor.TransitionTopic, "slack-notification", pubsub.SubscriptionConfig[*monitor.TransitionEvent]{ Handler: func(ctx context.Context, event *monitor.TransitionEvent) error { // Compose our message. msg := fmt.Sprintf("*%s is down!*", event.Site.URL) if event.Up { msg = fmt.Sprintf("*%s is back up.*", event.Site.URL) } // Send the Slack notification. return Notify(ctx, &NotifyParams{Text: msg}) }, }) ``` ## 8. Deploy your finished Uptime Monitor Deploy the finished Uptime Monitor, Slack integration included. ### Self-hosting Because we have added more infrastructure to our app, we need to [update the configuration](https://encore.dev/docs/self-host/configure-infra) in our `infra-config.json` to include the new Pub/Sub topic and subscription as well as how we should set the `SlackWebhookURL` secret. 🥐 Update your `infra-config.json` to reflect the new infrastructure. 🥐 Build a Docker image by running `encore build docker --config infra-config.json uptime:v2.0`. 🥐 Upload the Docker image to the cloud provider and run it. 🥐 As before, deploying is as simple as running: ```shell $ git add -A . $ git commit -m 'Add slack integration' $ git push encore ``` ## Conclusion You've built a working uptime monitoring system in a bit over 300 lines of code: * Three services (`site`, `monitor`, and `slack`) * Two databases, tracking the monitored sites and the check results * A cron job that checks every site every hour * A Pub/Sub topic decoupling the monitoring from the notifications * A Slack integration that subscribes to up/down transitions, with the webhook URL stored as a secret Next, build a [Slack bot](https://encore.dev/docs/tutorials/slack-bot), or see the infrastructure you can declare in code in the **Encore.ts:** [Encore.ts primitives](https://encore.dev/docs/ts/primitives)**Encore.go:** [Encore.go primitives](https://encore.dev/docs/go/primitives). Questions? Join us on [Discord](https://encore.dev/discord). --- Source: https://encore.dev/docs/tutorials/slack-bot Section: Getting started # Building a Slack bot > Learn how to build a Slack bot with an Encore backend In this tutorial you will create a Slack bot that brings the greatness of the `cowsay` utility to Slack! ![Slack Cowsay](https://encore.dev/assets/docs/cowsay.png "Slack bot") This is the end result: **Encore.ts:** _[Interactive editor showing the finished `slackBotTS` app — open this page in a browser to see it.]_ **Encore.go:** _[Interactive editor showing the finished `slackBot` app — open this page in a browser to see it.]_ > **Note** > > To make it easier to follow along, we've laid out a trail of croissants to guide your way. > Whenever you see a 🥐 it means there's something for you to do. ## 1. Create your Encore application 🥐 Create a new Encore application by running `encore app create` and select `Empty app` as the template. **Take a note of your app id, we'll need it in the next step.** ## 2. Create a Slack app 🥐 The first step is to create a new Slack app: 1. Head over to [Slack's API site](https://api.slack.com/apps) and create a new app. 2. When prompted, choose to create the app **from an app manifest**. 3. Choose a workspace to install the app in. 🥐 Enter the following manifest (replace `$APP_ID` in the URL below with your app id from above): ```yaml _metadata: major_version: 1 display_information: name: Encore Bot description: Cowsay for the cloud age. features: slash_commands: - command: /cowsay # Replace $APP_ID below url: https://staging-$APP_ID.encr.app/cowsay description: Say things with a flair! usage_hint: your message here should_escape: false bot_user: display_name: encore-bot always_online: true oauth_config: scopes: bot: - commands - chat:write - chat:write.public settings: org_deploy_enabled: false socket_mode_enabled: false token_rotation_enabled: false ``` Once created, we're ready to move on with implementing our Encore endpoint! ## 3. Implement the Slack endpoint Since Slack sends custom HTTP headers that we need to pay attention to, we're going to use a raw endpoint in Encore. For more information on this check out Slack's documentation on [Enabling interactivity with Slash Commands](https://api.slack.com/interactivity/slash-commands). **Encore.ts:** 🥐 In your Encore app, create a directory named `slack` containing a file named `encore.service.ts`. ```shell $ mkdir slack $ touch slack/encore.service.ts ``` 🥐 Add the following code to `slack/encore.service.ts`: ```ts -- slack/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("slack"); ``` This is how you create define services with Encore. Encore will now consider files in the `slack` directory and all its subdirectories as part of the `slack` service. 🥐 Create a file `slack/slack.ts` with the following contents: ```ts -- slack/slack.ts -- import { api } from "encore.dev/api"; import type { IncomingMessage } from "node:http"; // cowart is the formatting string for printing the cow art. const cowart = (msg: string) => `Moo! ${msg} `; export const cowsay = api.raw( { expose: true, path: "/cowsay", method: "*" }, async (req, resp) => { const body = await getBody(req); const text = new URLSearchParams(body).get("text"); const msg = cowart(text || "Moo!"); resp.setHeader("Content-Type", "application/json"); resp.end(JSON.stringify({ response_type: "in_channel", text: msg })); }, ); // Extract the body from an incoming request. function getBody(req: IncomingMessage): Promise { return new Promise((resolve) => { const bodyParts: any[] = []; req .on("data", (chunk) => { bodyParts.push(chunk); }) .on("end", () => { resolve(Buffer.concat(bodyParts).toString()); }); }); } ``` **Encore.go:** 🥐 In your Encore app, create a new directory named `slack` and create a file `slack/slack.go` with the following contents: ```go -- slack/slack.go -- // Service slack implements a cowsaw Slack bot. package slack import ( "encoding/json" "fmt" "net/http" ) // cowart is the formatting string for printing the cow art. const cowart = "Moo! %s" //encore:api public raw path=/cowsay func Cowsay(w http.ResponseWriter, req *http.Request) { text := req.FormValue("text") data, _ := json.Marshal(map[string]string{ "response_type": "in_channel", "text": fmt.Sprintf(cowart, text), }) w.Header().Set("Content-Type", "application/json") w.WriteHeader(200) w.Write(data) } ``` Let's try it out locally. 🥐 Start your app with `encore run` and then call it in another terminal: ```shell $ curl http://localhost:4000/cowsay -d 'text=Eat your greens!' {"response_type":"in_channel","text":"Moo! Eat your greens!"} ``` Looks great! 🥐 Next, let's deploy it to the cloud: ```shell $ git add -A . $ git commit -m 'Initial commit' $ git push encore ``` Once deployed, we're ready to try our Slack command! 🥐 Head over to the workspace you installed the app in and run `/cowsay Hello there`. You should see something like this: ![Cowsay](https://encore.dev/assets/docs/cowsay-wip.png "Cowsay (Work in Progress)") And just like that we have a fully working Slack integration. ## 4. Secure the webhook endpoint In order to get up and running quickly we ignored one important aspect for a production-ready Slack app: verifying that the webhook requests are actually coming from Slack. Let's do that now! The Slack documentation covers this really well on the [Verifying requests from Slack](https://api.slack.com/authentication/verifying-requests-from-slack) page. In short, what we need to do is: 1. Save a shared secret that Slack provides us 2. Use the secret to verify that the request comes from Slack, using HMAC (Hash-based Message Authentication Code). ### Save the shared secret Let's define a secret using Encore's secrets management functionality. **Encore.ts:** 🥐 Add this to your `slack.ts` file: ```ts HL slack/slack.ts 0:0 HL slack/slack.ts 2:2 -- slack/slack.ts -- import { secret } from "encore.dev/config"; const slackSigningSecret = secret("SlackSigningSecret"); ``` **Encore.go:** 🥐 Add this to your `slack.go` file: ```go -- slack/slack.go -- var secrets struct { SlackSigningSecret string } ``` 🥐 Head over to the configuration section for your Slack app (go to [Your Apps](https://api.slack.com/apps) → select your app → Basic Information). 🥐 Copy the **Signing Secret** and then run `encore secret set --type prod SlackSigningSecret` and paste the secret. 🥐 For development you will also want to set `encore secret set --type dev,local,pr SlackSigningSecret`. You can use the same secret value or a placeholder value. ### Compute the HMAC **Encore.ts:** TypeScript makes computing HMAC very straightforward, but it's still a fair amount of code. **Encore.go:** Go makes computing HMAC very straightforward, but it's still a fair amount of code. 🥐 Add a few more imports to your file, so that it reads: **Encore.ts:** ```ts -- slack/slack.ts -- import { createHmac, timingSafeEqual } from "node:crypto"; import type { IncomingHttpHeaders } from "http"; ``` **Encore.ts:** 🥐 Next, we'll add the `verifySignature` function: ```ts -- slack/slack.ts -- // Verifies the signature of an incoming request from Slack. const verifySignature = async function ( body: string, headers: IncomingHttpHeaders, ) { const requestTimestampSec = parseInt( headers["x-slack-request-timestamp"] as string, ); const signature = headers["x-slack-signature"] as string; if (Number.isNaN(requestTimestampSec)) { throw new Error( `Failed to verify authenticity: header x-slack-request-timestamp did not have the expected type (${requestTimestampSec})`, ); } // Calculate time-dependent values const nowMs = Date.now(); const requestTimestampMaxDeltaMin = 5; const fiveMinutesAgoSec = Math.floor(nowMs / 1000) - 60 * requestTimestampMaxDeltaMin; // Enforce verification rules // Rule 1: Check staleness if (requestTimestampSec < fiveMinutesAgoSec) { throw new Error( `Failed to verify authenticity: x-slack-request-timestamp must differ from system time by no more than ${requestTimestampMaxDeltaMin} minutes or request is stale`, ); } // Rule 2: Check signature // Separate parts of signature const [signatureVersion, signatureHash] = signature.split("="); // Only handle known versions if (signatureVersion !== "v0") { throw new Error(`Failed to verify authenticity: unknown signature version`); } // Compute our own signature hash const hmac = createHmac("sha256", slackSigningSecret()); hmac.update(`${signatureVersion}:${requestTimestampSec}:${body}`); const ourSignatureHash = hmac.digest("hex"); if ( !signatureHash || !timingSafeEqual( Buffer.from(signatureHash, "utf8"), Buffer.from(ourSignatureHash, "utf8"), ) ) { throw new Error(`Failed to verify authenticity: signature mismatch`); } }; ``` **Encore.go:** ```go -- slack/slack.go -- import ( "crypto/hmac" "crypto/sha256" "encoding/hex" "encoding/json" "fmt" "io/ioutil" "net/http" "net/url" "strconv" "strings" "time" "encore.dev/beta/errs" "encore.dev/rlog" ) ``` 🥐 Next, we'll add the `verifyRequest` function: ```go -- slack/slack.go -- // verifyRequest verifies that a request is coming from Slack. func verifyRequest(req *http.Request) (body []byte, err error) { eb := errs.B().Code(errs.InvalidArgument) body, err = ioutil.ReadAll(req.Body) if err != nil { return nil, eb.Cause(err).Err() } // Compare timestamps to prevent replay attack ts := req.Header.Get("X-Slack-Request-Timestamp") threshold := int64(5 * 60) n, _ := strconv.ParseInt(ts, 10, 64) if diff := time.Now().Unix() - n; diff > threshold || diff < -threshold { return body, eb.Msg("message not recent").Err() } // Compare HMAC signature sig := req.Header.Get("X-Slack-Signature") prefix := "v0=" if !strings.HasPrefix(sig, prefix) { return body, eb.Msg("invalid signature").Err() } gotMac, _ := hex.DecodeString(sig[len(prefix):]) mac := hmac.New(sha256.New, []byte(secrets.SlackSigningSecret)) fmt.Fprintf(mac, "v0:%s:", ts) mac.Write(body) expectedMac := mac.Sum(nil) if !hmac.Equal(gotMac, expectedMac) { return body, eb.Msg("bad mac").Err() } return body, nil } ``` > **Note** > > As you can see, this function needs to consume the whole HTTP body in order to compute the HMAC. > > This breaks the use of `req.FormValue("text")` that we used earlier, since it relies on reading the HTTP body. That's the reason we're returning the body from `verifyRequest`, so that we can parse the form values from that directly instead. We're now ready to verify the signature. **Encore.ts:** 🥐 Update the `cowsay` function to look like this: ```ts HL slack/slack.ts 5:12 -- slack/slack.ts -- export const cowsay = api.raw( { expose: true, path: "/cowsay", method: "*" }, async (req, resp) => { const body = await getBody(req); try { await verifySignature(body, req.headers); } catch (err) { const e = err as Error; resp.statusCode = 500; resp.end(e.message); return; } const text = new URLSearchParams(body).get("text"); const msg = cowart(text || "Moo!"); resp.setHeader("Content-Type", "application/json"); resp.end(JSON.stringify({ response_type: "in_channel", text: msg })); }, ); ``` **Encore.go:** 🥐 Update the `Cowsay` function to look like this: ```go -- slack/slack.go -- //encore:api public raw path=/cowsay func Cowsay(w http.ResponseWriter, req *http.Request) { body, err := verifyRequest(req) if err != nil { errs.HTTPError(w, err) return } q, _ := url.ParseQuery(string(body)) text := q.Get("text") data, _ := json.Marshal(map[string]string{ "response_type": "in_channel", "text": fmt.Sprintf(cowart, text), }) w.Header().Set("Content-Type", "application/json") w.WriteHeader(200) w.Write(data) } ``` ## 5. Put it all together and deploy Finally we're ready to put it all together. **Encore.ts:** 🥐 Add the `cowart` in `slack.ts` like so: ```ts -- slack/slack.ts -- const cowart = (msg: string) => ` \`\`\` +-${"-".repeat(msg.length)}-+ | ${msg} | +-${"-".repeat(msg.length)}-+ \\ __n__n__ .------\`-\\00/-' / ## ## (oo) / \\## __ ./ |//YY \\|/ ||| ||| \`\`\` `; ``` **Encore.go:** 🥐 Add the `cowart` like so: ```go -- slack/slack.go -- const cowart = ` ________________________________________ < %- 38s > ---------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || ` ``` 🥐 Finally, let's commit our changes and deploy it: ```shell $ git add -A . $ git commit -m 'Verify webhook requests and improve art' $ git push encore ``` 🥐 Once deployed, head back to Slack and run `/cowsay Hello there`. If everything is set up correctly, you should see: ![Slack Cowsay](https://encore.dev/assets/docs/cowsay.png "Slack bot") And there we go, a production-ready Slack bot in less than 100 lines of code. Well done! ### Celebrate with fireworks Now that your app is running in the cloud, let's celebrate with some fireworks: 🥐 In the Cloud Dashboard, open the Command Menu by pressing **Cmd + K** (Mac) or **Ctrl + K** (Windows/Linux). _From here you can easily access all Cloud Dashboard features and for example jump straight to specific services in the Service Catalog or view Traces for specific endpoints._ 🥐 Type `fireworks` in the Command Menu and press enter. Sit back and enjoy the show! ![Fireworks](https://encore.dev/assets/docs/fireworks.jpg) --- Source: https://encore.dev/docs/faq Section: Getting started # Frequently Asked Questions > The essentials about building and running applications with Encore ## About Encore ### What is Encore? Encore is a backend development platform for TypeScript and Go. You define APIs, services, and infrastructure resources in application code. Encore uses those declarations to provide local infrastructure and development tooling, validate the application, and provision matching resources when you deploy. [Understanding Encore](https://encore.dev/docs/understanding-encore) explains how those pieces fit together. ### Is Encore open source? Yes. The Encore SDKs, parser, compiler, CLI, and runtime are open source under the Mozilla Public License 2.0. The managed cloud platform is a separate service built around those tools. See [Open Source](https://encore.dev/docs/community/open-source) for licensing and contribution details. ### Do I need to use Encore's managed platform? No. The open source tooling provides local development, testing, and production Docker builds. You can [build a Docker image](https://encore.dev/docs/self-host/build), provision infrastructure yourself, and supply its connection details with an [infrastructure configuration file](https://encore.dev/docs/self-host/configure-infra). The managed platform adds automated cloud provisioning, deployments, environments, preview environments, and operational tooling. ## Languages and application code ### Which languages does Encore support? Encore supports TypeScript and Go through the [Encore.ts](https://encore.dev/docs/ts) and [Encore.go](https://encore.dev/docs/go) SDKs. An Encore application currently uses one of the two languages; separate applications can communicate through APIs. ### Does Encore support Python? Encore's native SDKs currently support TypeScript and Go. Python services can run separately and communicate with Encore applications through APIs, but they do not receive Encore's native resource declarations, static analysis, or local development integration. ### Can I use normal TypeScript and Go libraries? Yes. Encore applications use the standard TypeScript/Node.js and Go ecosystems. Static analysis places requirements on how Encore resources and APIs are declared, but it does not otherwise restrict how you structure application logic or which compatible libraries you use. Some libraries that own the HTTP server, process lifecycle, or infrastructure connection may need an Encore-specific integration. Check the relevant SDK documentation when adopting a framework or runtime plugin. ## Infrastructure and deployment ### Which infrastructure can Encore manage? Encore has built-in primitives for SQL databases, Pub/Sub, object storage, caches, cron jobs, and secrets. The same declaration receives a local implementation during development and a managed implementation in a cloud environment. Infrastructure without an Encore primitive can run alongside the application as an external dependency. See the [Encore.ts](https://encore.dev/docs/ts) and [Encore.go](https://encore.dev/docs/go) overviews for the supported primitives and [Understanding Encore](https://encore.dev/docs/understanding-encore#infrastructure-outside-the-model) for the boundary. ### Where can an Encore application run? Encore's managed platform can deploy production applications and supported infrastructure into your own AWS or GCP account. Development and preview environments can run on Encore-managed infrastructure, and Enterprise teams can host previews in their own cloud. For self-hosting, Encore builds a standard Docker image that you can deploy wherever its runtime and infrastructure requirements are available. ### Can I deploy to Azure? Encore's managed platform currently provisions and deploys production environments on AWS and GCP. You can self-host an Encore Docker image on Azure, but you manage the deployment and infrastructure yourself; Encore does not provision Azure services natively. ### Can Encore use infrastructure I already have? Yes, with resource-specific boundaries. Managed environments can connect supported existing databases, buckets, topics, cloud projects, and Kubernetes clusters. Self-hosted environments can map Encore declarations to infrastructure you provision yourself. Start with [Coming from an IaC tool](https://encore.dev/docs/platform/migration/from-iac) for the ownership models and supported import workflows. ## Adopting Encore ### Can I migrate an existing backend incrementally? Yes. A common approach is to move one service or route group at a time while the existing system continues to handle everything else. Infrastructure can move separately: supported resources may be imported, recreated and transferred, or kept as external dependencies. See [Migrating an existing system to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore) and the product-specific migration guides. ### How do I get started or ask for help? Use the [Quickstart guide](https://encore.dev/docs/quick-start) to create and run an application. For questions, join the [Encore community on Discord](https://encore.dev/discord) or browse the [community documentation](https://encore.dev/docs/community/get-involved). --- Source: https://encore.dev/docs/platform/test-statistics Section: Building # Test Statistics > Every test result from every build, in one place When Encore Cloud builds your application, it runs your tests and records the result of every test case. The **Tests** page in the [Cloud Dashboard](https://app.encore.dev) turns those results into answers: what is failing right now, which tests are flaky, where the time in your test suite goes, and which commit made a test slower. **Encore.ts:** Encore.ts applications publish their test results by writing a JUnit XML report, which takes a few lines of test runner configuration. See [Publishing test results](#publishing-test-results). **Encore.go:** Encore.go applications report their test results automatically. See [Publishing test results](#publishing-test-results). Encore Cloud can also record which lines of your code the tests run. See [Test Coverage](https://encore.dev/docs/platform/test-coverage). ![The Tests overview](https://encore.dev/assets/docs/test-stats-overview.png "The Tests overview") ## Publishing test results Every build runs your tests as part of its build & test phase, and Encore Cloud reads the results from JUnit XML reports. **Encore.ts:** The tests run with the `test` script in your `package.json`, so your test runner decides what gets reported. When the tests run in Encore Cloud, the `ENCORE_TEST_REPORT_DIR` environment variable is set to a directory. Configure your test runner to write a JUnit XML report into it. Encore Cloud reads every `.xml` file in that directory and in its immediate subdirectories. With [Vitest](https://vitest.dev), turn on the built-in `junit` reporter when `ENCORE_TEST_REPORT_DIR` is set. Starting from the `vite.config.ts` in the [testing guide](https://encore.dev/docs/ts/develop/testing#setting-up-vitest), add a `test` section: ```ts -- vite.config.ts -- /// import { defineConfig } from "vite"; import path from "path"; // Set by Encore Cloud when it runs your tests; unset locally. const reportDir = process.env.ENCORE_TEST_REPORT_DIR; export default defineConfig({ resolve: { alias: { "~encore": path.resolve(__dirname, "./encore.gen"), }, }, test: { // Publish a JUnit report to Encore Cloud, and keep the usual output in the build log. reporters: reportDir ? ["default", "junit"] : ["default"], outputFile: reportDir ? { junit: path.join(reportDir, "junit.xml") } : undefined, }, }); ``` Running `encore test` locally is unaffected: the variable is not set there, so Vitest prints its usual output and writes no report. Any test runner with a JUnit reporter works the same way: point its output at `ENCORE_TEST_REPORT_DIR`. For example, with Jest, add [`jest-junit`](https://github.com/jest-community/jest-junit) as a reporter and set `JEST_JUNIT_OUTPUT_DIR` to the value of `ENCORE_TEST_REPORT_DIR`. **Encore.go:** > **Note** > > Encore.go applications report test results automatically. There is nothing to configure. Encore runs your tests with `go test -json` and turns the results into a report. The build log still shows the usual `go test -v` output. If a build writes no report, its run is still listed, as **No report**, together with whether the tests passed. ## Tests overview Open your app in the Cloud Dashboard and select **Tests** in the main navigation. The overview covers the last 7, 30 or 90 days: - **Pass rate**, **Runs**, **Tests** and **Avg duration**, each compared with the period before. Skipped tests don't count toward the pass rate. - **Runs per day**, split into passed and failed runs. - **Needs attention** lists the tests to look at first, each with its last 20 results: - **Broken** tests failed in their most recent run, after passing earlier in the period. - **Flaky** tests both passed and failed during the period, and switched between the two at least twice. - **Slowest tests** ranks tests by their 95th percentile duration. - **Recent runs** lists the latest runs with their commit, the environments the build was deployed to, and the result. ## Test runs Every build that runs tests produces a test run. Open a run from **Recent runs** or from the test step of a deploy. ![A test run](https://encore.dev/assets/docs/test-stats-run.png "A test run") The run compares itself with an earlier run on the same branch or environment: - **New failures** are tests that fail now but passed last time. - **Still failing** are tests that failed in both runs. - **Fixed** are tests that failed last time and pass now. - **Tests added** counts tests that are new in this run, and how many were removed. **Compare durations** opens the two runs in [Compare runs](#compare-runs). When your app measures coverage, the run also shows its coverage, and its **Coverage** tab shows whether the lines the commit changed are tested. See [Test Coverage](https://encore.dev/docs/platform/test-coverage#in-a-test-run). Below the comparison, the run's tests are grouped into **Failed**, **Passed**, **Skipped** and **Slowest**, and you can filter them by test name. A failed test shows its failure message, and you can switch between the failure and the test's stdout and stderr. Output is kept for failed tests only. Each test also shows its last 20 results and links to its history. ### In a deploy The deploy page shows the same results in its **Test** step: how many tests failed, passed and were skipped, the first failures with their output, and whether the failing tests passed on the previous run. **Open test run** takes you to the full run. ![The Test step of a deploy](https://encore.dev/assets/docs/test-stats-deploy.png "The Test step of a deploy") ## Test history Select a test anywhere on the Tests pages to see its history over the last 30 days: how often it ran, its failure rate, its median (p50) and 95th percentile (p95) duration, and when it last failed. A test is marked **Flaky** or **Failing** when it is one. ![The history of a test](https://encore.dev/assets/docs/test-stats-history.png "The history of a test") **Every result** shows each run of the test as one bar, and the height of the bar is how long the test took. When every failure took much longer than a passing run usually does, the page points out that the failures may be timeouts rather than wrong results. **Failures** lists each failure with its message, commit, environment and duration. ## Durations **Durations** shows where the time in your test suite goes, and which tests are getting slower. Every number is compared with the period before. ![Test durations](https://encore.dev/assets/docs/test-stats-durations.png "Test durations") - **Test Job, p50** is the median wall time of the test step of your builds. **Test Job duration** charts it over time together with its p95. - **Time in tests** adds up every test's average duration. That is how long a run would take if the tests ran one after another, which makes it the time you save by speeding tests up. It is usually longer than the test step itself, because suites run in parallel. - **Slow tests** counts the tests with a p50 of one second or more. - **How long tests take** groups the tests by their p50, and **Where the time goes** shows the share of the total time that the slowest tests take. ![Time by suite and tests getting slower](https://encore.dev/assets/docs/test-stats-getting-slower.png "Time by suite and tests getting slower") **Time by suite** breaks the time down by suite. **Getting slower** lists the tests whose p50 went up the most compared with the period before. ![All tests by duration](https://encore.dev/assets/docs/test-stats-all-tests.png "All tests by duration") **All tests** lists every test with its runs, p50, p95, slowest run, share of the total time, and how its p50 changed. Search by test, suite or file, and narrow the list down to: - **Over 1s**: tests with a p50 of one second or more. - **Getting slower**: tests whose p50 went up compared with the period before. - **Erratic**: tests whose p95 is at least twice their p50, and at least one second. ## Regressions A regression is a commit after which one or more tests became consistently and significantly slower. **Regressions** lists them, starting with the ones that are still open. ![Regressions](https://encore.dev/assets/docs/test-stats-regressions.png "Regressions") A test has regressed when its p50 went up by at least 30% **and** at least one second, and stayed there. Encore Cloud looks for a step up between two runs, with at least five passing runs on each side. A test that slowly drifts slower over many commits is not pinned on any one commit. It shows up under **Getting slower** on the Durations page instead. Only passing runs count, because failures are often timeouts. Regressions are found on your app's main line: the runs of builds that were deployed to a production or other persistent environment, in the order they ran. Builds that only reach [preview environments](https://encore.dev/docs/platform/deploy/preview-environments) are left out, since their code is not on the main line yet. Encore Cloud checks for new regressions a couple of minutes after each run. A regression is **resolved** once every slower test is fast again, or has not run in the last 10 main-line runs because it was removed or renamed. If a slowdown is expected, for example because a test now does more work, **Dismiss** it with a reason. A dismissed regression can be reopened. Dismissing and reopening requires the Admin or Writer role. ## Compare runs **Compare runs** compares the test durations and results of two sets of runs, such as the last 20 runs before a commit against the runs from that commit on. Comparing sets of runs rather than two single runs keeps one noisy run from deciding the outcome. Open it with **Compare runs** on a regression, which compares the runs before and after its commit, or with **Compare durations** on a test run. ![Comparing two sets of runs](https://encore.dev/assets/docs/test-stats-compare.png "Comparing two sets of runs") Each test is labeled by how its p50 changed: - **Regressed**: slower by at least 30% and one second, the same threshold as for regressions. - **Slower** or **Faster**: changed by more than the normal run-to-run variation, but below that threshold. - **New** or **Removed**: only in the runs after, or only in the runs before. Changes within the normal run-to-run variation are hidden until you select **Show all**. With at least three passing results on each side, a change must be statistically significant and at least 10ms and 5%. With fewer results, it must be at least 50ms and 20%. ## Cached test results Go caches the results of test packages that haven't changed, and reports them as `(cached)` instead of running them again. Encore Cloud records these results as cached: - A cached result still counts as a run, and as a pass or a failure. - A cached result's duration comes from an earlier run, so it's left out of every duration statistic, from percentiles and regressions to comparisons. - Cached results are left out of flakiness too, because a replayed pass is not another attempt. - In totals such as **Time in tests**, a cached result counts as taking no time, because it didn't. Cached results are marked **cached** where their duration would be shown, and runs show how many of their results were cached. ## Access and retention Anyone who can view the app's code can view its test results. That includes the Admin, Writer and Reader roles. Failure output can contain source code, so test results are never visible to someone who can't see the code. Test results are kept for one year. The history of a single test covers up to the last 90 days. --- Source: https://encore.dev/docs/platform/test-coverage Section: Building # Test Coverage > Which lines your tests run, and where new code goes untested When Encore Cloud runs your tests, it can also record which lines of your code they run. The **Coverage** page in the [Cloud Dashboard](https://app.encore.dev) shows how much of your application your tests run, which files and services have the least coverage, how coverage changes over time, and whether the lines each commit changes are tested. **Encore.ts:** Encore.ts applications publish coverage by writing an lcov report, which takes a few lines of test runner configuration. See [Measuring coverage](#measuring-coverage). **Encore.go:** Encore.go applications measure coverage automatically. See [Measuring coverage](#measuring-coverage). Coverage builds on [Test Statistics](https://encore.dev/docs/platform/test-statistics): it is recorded for the same test runs, and it's on the same **Tests** pages. ![The Coverage page](https://encore.dev/assets/docs/test-coverage-overview.png "The Coverage page") ## Measuring coverage Coverage is measured per line. A line counts when it contains code that can run, and it is covered when at least one test ran it during the run. Blank lines and comments don't count. Encore Cloud also records how many times each line ran. **Encore.ts:** Your test runner measures coverage. When the tests run in Encore Cloud, the `ENCORE_TEST_REPORT_DIR` environment variable is set to a directory, the same one you write [JUnit reports](https://encore.dev/docs/platform/test-statistics#publishing-test-results) to. Configure your test runner to write an lcov report into it. Encore Cloud reads every `lcov.info` and `.lcov` file in that directory and in its immediate subdirectories. With [Vitest](https://vitest.dev), install the V8 coverage provider, using the same version as Vitest: ```shell $ npm install --save-dev @vitest/coverage-v8 ``` Then turn on coverage in your `vite.config.ts` when `ENCORE_TEST_REPORT_DIR` is set, next to the JUnit reporter: ```ts -- vite.config.ts -- /// import { defineConfig } from "vite"; import path from "path"; // Set by Encore Cloud when it runs your tests; unset locally. const reportDir = process.env.ENCORE_TEST_REPORT_DIR; export default defineConfig({ resolve: { alias: { "~encore": path.resolve(__dirname, "./encore.gen"), }, }, test: reportDir ? { reporters: ["default", "junit"], outputFile: { junit: path.join(reportDir, "junit.xml") }, coverage: { enabled: true, provider: "v8", reporter: ["lcovonly"], reportsDirectory: path.join(reportDir, "coverage"), // Publish coverage for runs with failing tests too. reportOnFailure: true, // Your services' directories, so files no test imports count as untested. include: ["monitor/**", "site/**"], }, } : {}, }); ``` Some things to know: - **`reportOnFailure`**: by default, Vitest writes no coverage when a test fails. Encore Cloud records coverage for failed runs too, so turn it on. - **`include`**: without it, Vitest only reports the files your tests import. List your services' directories so that files no test touches show up as untested, and leave out code that isn't tested here, such as a frontend. - **`lcovonly`** writes just the lcov file. The `lcov` reporter also writes an HTML report, which Encore Cloud doesn't use. Running `encore test` locally is unaffected: the variable is not set there, so Vitest doesn't measure coverage. Any test runner that writes lcov works the same way: point its output at `ENCORE_TEST_REPORT_DIR`. For example, with Jest, set `collectCoverage` to `true`, `coverageReporters` to `["lcovonly"]` and `coverageDirectory` to the value of `ENCORE_TEST_REPORT_DIR`. File paths in the report can be absolute or relative to your app's root directory. Files outside your repository, under `node_modules`, or no longer in the repository are left out. **Encore.go:** > **Note** > > Encore.go applications measure coverage automatically. There is nothing to configure. Encore runs your tests with Go's coverage measurement turned on, package by package: - A package's coverage comes from its own tests. Code that a test in another package happens to call is not counted as covered. - Packages without tests are still included, with none of their lines covered, so code you haven't written tests for shows up as untested. - When Go replays a package's [cached test results](https://encore.dev/docs/platform/test-statistics#cached-test-results), its coverage is replayed with them. - Generated files, which start with a `// Code generated ... DO NOT EDIT.` comment, are left out. ### Limits A run's coverage is left out if it is larger than 8 MiB compressed, or if it would make the run's test reports larger than 32 MiB. The test results are still recorded. The build log says when coverage was left out. ## The Coverage page Open **Tests** in the Cloud Dashboard and select **Coverage**. The page covers the last 7, 30 or 90 days. The coverage of your app is the coverage of the latest run on the main line: the runs of builds that were deployed to a production or other persistent environment. If your app has no such environment, every run counts. Only runs where the tests passed count, because a test that fails part-way through, or a package that doesn't build, leaves code looking untested when it isn't. - **Line coverage** is the share of your app's lines that tests run, and how it changed over the period. - **Changed lines** is the share of the lines changed during the period that tests run. See [Changed lines](#changed-lines). - **Untested files** counts the files where no test runs a single line. - **Lowest service** is the service with the least coverage. **Coverage over time** charts the coverage of every run on the main line. The dots show the coverage of the lines each run's commit changed. The run after which coverage fell the most is pointed out, when it fell by at least half a point. **Coverage by service** lists your services with the least covered first, and how each one's coverage changed over the period. Select a service to see it in the code map. ### Code map The **Code map** shows your app's code as rings of directories and files. The size of each piece is how many lines it has that can run, and its colour is its coverage, from red below 60% to grey at 90% and above. That makes large, poorly covered parts of your code easy to spot. Hover over a piece to see its coverage. Select a directory to zoom in on it, and select the centre to go back up. Selecting a file opens it. ### Files The file list follows the code map: it shows the files in the directory you have zoomed in on, or in your whole app. For each file, it shows how many of its lines can run, how many of them are untested, where in the file the untested lines are, and its coverage. The **30d** column charts how the file's coverage changed over the period, and **Change** is the difference. Files added during the period are marked as **new**. Narrow the list down to: - **Under 60%**: files with less than 60% coverage. - **Untested**: files where no test runs a single line. - **Changed in 30 days**: files changed on the main line during the period. Sort the list by file, lines, untested lines or coverage, and filter it by file name. Select a file to open it. ## A file's coverage The page of a file shows its source with the coverage of each line, from the latest run on the main line. When you open it from a test run, it shows that run's coverage instead. ![The coverage of a file](https://encore.dev/assets/docs/test-coverage-file.png "The coverage of a file") Each line that can run shows how many times tests ran it. Lines no test runs are marked in red. Long stretches of tested lines are folded away, so the untested code is what you see first. Show only the **Untested** lines, or the lines **Changed in this run**. **In this file** lists the file's functions, top to bottom, with how many of their lines tests run. **Over time** charts the file's coverage on the main line. ## Changed lines Overall coverage changes slowly, which makes it a poor signal for whether new code is tested. Encore Cloud therefore also measures the coverage of the lines each commit changed, compared with an earlier commit: - For a build on the main line, the commit of the previous run on the main line, so that the changes of every commit since are counted. - For other builds, the commit where its branch left the main line, so that a pull request is measured by everything it changes. If no such commit is found within the last 300 commits, the build's commit is compared with its parent instead. Renamed files are followed. If the repository can't be read, the run is recorded without changed-line coverage. ## In a test run A test run shows its coverage in its header, together with the coverage of the lines its commit changed, and compares its coverage with the earlier run it compares itself with. ![The Coverage tab of a test run](https://encore.dev/assets/docs/test-coverage-run.png "The Coverage tab of a test run") The **Coverage** tab starts with the files the run's commit changed, the ones with the most untested changed lines first. Expand a file to see its changed lines that no test runs, or select **Open file** to see the whole file in this run. **All files** lists every file in the run, the same way as on the [Coverage page](#files). When an app measures coverage, the **Tests** overview also shows its line coverage, and **Recent runs** shows the coverage of each run. ## Access and retention Coverage follows the same rules as [test results](https://encore.dev/docs/platform/test-statistics#access-and-retention): anyone who can view the app's code can view its coverage. A file's page shows its source code, read from your repository at the run's commit. Coverage is kept for one year. The history of a single file covers up to the last 90 days. --- Source: https://encore.dev/docs/app-structure Section: Building # App Structure > Design the services and relationships in an Encore application Encore uses a monorepo design: one application contains the services and infrastructure resources for your backend. Encore uses these services and their relationships to build the [application model](https://encore.dev/docs/understanding-encore), which drives local development, infrastructure provisioning, IAM, tracing, and the [architecture diagram](https://encore.dev/docs/architecture). Keep the backend in one Encore application where practical so the model covers the whole backend. Existing systems can remain separate and connect through APIs or generated clients during an incremental [migration to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore). ## Monolith or microservices Encore supports both monoliths and microservices. A new application should generally start with one service. Direct function calls and database transactions are easier to change than interfaces between services, and directories or Go subpackages can organize the code without adding a service boundary. Consider an application that accepts orders, reserves inventory, and sends confirmations. The first version keeps all three capabilities in an `orders` service. **Encore.ts:** In TypeScript, `encore.service.ts` defines the service: ```ts // orders/encore.service.ts import { Service } from "encore.dev/service"; export default new Service("orders"); ``` Encore includes the directory and its subdirectories in the service. A single-service application can place this file at the application root: ```text my-app/ ├── encore.app ├── package.json ├── encore.service.ts // defines the service root ├── orders.ts ├── inventory.ts ├── notifications.ts ├── db.ts └── migrations/ ``` Use a single root-level `package.json` for the Encore application. Shared workspace packages that require compilation must be built before Encore parses the application; the [Turborepo](https://encore.dev/docs/ts/develop/monorepo/turborepo) and [Nx](https://encore.dev/docs/ts/develop/monorepo/nx) guides show how to configure this. See [Defining services](https://encore.dev/docs/ts/primitives/services). **Encore.go:** In Go, a package containing at least one Encore API is a service: ```go package orders import "context" //encore:api private method=POST path=/orders/:id/cancel func Cancel(ctx context.Context, id string) error { // ... return nil } ``` A single-service application can keep the service package at the application root: ```text my-app/ ├── encore.app ├── go.mod ├── orders.go ├── inventory.go ├── notifications.go └── migrations/ ``` The service can contain internal subpackages, but those packages cannot define Encore APIs. See [Defining services](https://encore.dev/docs/go/primitives/services) and [service structs](https://encore.dev/docs/go/primitives/service-structs). Order placement and inventory reservation may update the same database transaction. Keeping them in one service preserves that transaction and avoids a network call in the request path. Sending a confirmation can run after the order succeeds, but this alone does not require another service; a Pub/Sub subscriber can belong to the same service. ## Add a service when the requirements change Suppose confirmations expand into a notification capability used by orders, billing, and account security. It uses an external provider, has its own retry behavior, and must remain available when order API instances fail: ```text orders service ├── owns orders and inventory ├── places orders and reserves stock synchronously └── publishes order events │ ▼ notifications service ├── subscribes to order events ├── owns notification preferences and delivery records └── calls the external notification provider ``` The application now has one directory or package per service: **Encore.ts:** ```text my-app/ ├── encore.app ├── package.json ├── orders/ │ ├── encore.service.ts // defines the orders service root │ ├── api.ts │ ├── inventory.ts │ ├── events.ts │ └── migrations/ └── notifications/ ├── encore.service.ts // defines the notifications service root ├── subscriptions.ts └── migrations/ ``` **Encore.go:** ```text my-app/ ├── encore.app ├── go.mod ├── orders/ │ ├── api.go │ ├── inventory.go │ ├── events.go │ └── migrations/ └── notifications/ ├── subscriptions.go └── migrations/ ``` Notifications now form a cohesive capability with their own data and failure behavior. Orders, billing, and account security can all use that capability without owning notification logic. ### When a service boundary helps - The service owns a cohesive capability with distinct operations and data. - Failure isolation, scaling, or security requirements differ from the rest of the application. - An existing external system already provides a network interface. Keep behavior together when most changes affect both sides, one transaction must cover the operation, or neither side can do useful work alone. If the goal is only code organization, use modules, directories, or Go subpackages. Separating `handlers`, `business-logic`, and `database` into services would replace local calls with network calls without giving each service a distinct responsibility. ## Choose how services communicate Suppose payment processing later moves into a `billing` service: - The orders service uses a typed service API to request payment because it needs the result before confirming the order. Calls retain type safety and IDE autocomplete across the service boundary. See service calls in [TypeScript](https://encore.dev/docs/ts/primitives/api-calls) and [Go](https://encore.dev/docs/go/primitives/api-calls). - The orders service publishes an event for notifications because the response does not depend on delivery. Other consumers can react to the same event independently. See Pub/Sub in [TypeScript](https://encore.dev/docs/ts/primitives/pubsub) and [Go](https://encore.dev/docs/go/primitives/pubsub). Pub/Sub workflows are eventually consistent. Messages can also be redelivered in some failure cases, including with exactly-once delivery. Make a handler idempotent when processing the same event more than once would produce an incorrect result. Long chains of synchronous calls make a request depend on every service in the chain. Ordered events that imitate synchronous calls add eventual consistency without making the services independent. ### Database changes and events A database transaction and a Pub/Sub publish are separate operations. If both must succeed together, account for a failure between them. Encore Go provides a [transactional Pub/Sub outbox](https://encore.dev/docs/go/how-to/pubsub-outbox) that records an event in the database transaction and publishes it after the transaction commits. TypeScript applications and Go applications not using the outbox require an application-specific consistency mechanism. ## Own and share data The orders service owns order and inventory data. The notifications service uses its API or reacts to its events instead of modifying those tables directly. Encore also supports direct database access from multiple services through `SQLDatabase.named("name")` in TypeScript and `sqldb.Named("name")` in Go. Shared access can be appropriate for reporting, incremental migrations, or cases where another service interface adds more complexity than it removes. A reporting service can initially read the orders database while the orders service retains responsibility for migrations and writes: ```text orders service ── owns schema and writes ──► orders database reporting service ── read-only access ─────► orders database ``` If reporting queries later interfere with transactional work, the reporting service can own a separate read model populated from order events. Reports may lag behind orders, but reporting queries no longer compete with order transactions. Define migration ownership and write access before sharing a database. Shared schema access couples readers to schema changes even when the owning service's API remains unchanged. ## Services and processes An Encore service is a code and API boundary, while [process allocation](https://encore.dev/docs/platform/deploy/environments#configuring-process-allocation) controls how services run in each environment: - **Single process:** all services run together, reducing runtime overhead and keeping service calls within one process. - **Separate processes:** each service runs independently for stronger isolation and independent scaling. The notifications example requires separate processes in production to isolate delivery from order API failures. A development environment can still run all services in one process. Use service boundaries for application responsibilities and process allocation for runtime isolation and scaling. ## Define the external interface The orders API may be public or authenticated. Operations used only by other services can remain private. Configure authentication and cross-origin policy at the API gateway. Encore propagates authenticated user data to service calls. The service that owns a record decides who may act on it. See the authentication guides for [TypeScript](https://encore.dev/docs/ts/develop/auth) and [Go](https://encore.dev/docs/go/develop/auth). ## Organize larger applications As the application adds services, a flat directory can become difficult to navigate. Services cannot be nested, but related services can be grouped into system directories: ```text my-app/ ├── encore.app ├── commerce/ │ ├── orders/ │ ├── reporting/ │ └── payments/ ├── identity/ │ ├── users/ │ └── auth/ └── communications/ ├── notifications/ └── templates/ ``` A system is an organizational convention rather than an Encore resource. Moving the orders service under `commerce`, for example, does not change its APIs or deployment behavior. ## Review a proposed boundary - Does the service own a cohesive application capability? - Which operations and schema changes does it own? - Which calls or events cross the boundary? - What does a caller do when the service is unavailable? - Does the caller need an immediate result? - Would most changes still require modifying both services? - Can process allocation meet the scaling or isolation requirement without another service? To split an existing service, move the related operations and data together, replace direct calls across the new boundary with APIs or events, and define data access during the transition. Inspect the result in the [architecture diagram](https://encore.dev/docs/architecture) and test calls that now cross the boundary. Public API paths do not need to change when their implementation moves to another service. See [Break a monolith into microservices](https://encore.dev/docs/break-up-monolith) for a complete example. --- Source: https://encore.dev/docs/mono-vs-multi-repo Section: Building # Mono vs Multi Repo > How to structure your frontend and backend Encore is not opinionated about if you have your backend and frontend code in the same repo or not. Pick the approach that fits your application best. ## Monorepo If you use a monorepo then it is often a good idea to place your backend and frontend in separate folders in the root of your repo, like so: ``` /my-app ├── backend │ ├── encore.app │ ├── package.json // Backend dependencies (go.mod for Encore.go) │ └── ... └── frontend ├── package.json // Frontend dependencies └── ... ``` This way, you can keep your frontend and backend dependencies separate, while still having the codebases in the same repository. If you are using the Encore platform for deployment, remember to configure the "Root Directory" in app settings in the [Encore platform dashboard](https://app.encore.dev) to point to where you have your `encore.app` file. **Related example:** [encoredev/nextjs-starter](https://github.com/encoredev/nextjs-starter) Next.js + Encore Starter, separated into frontend and backend folders. You can also have a monorepo where the `encore.app` file is in the root of the repo, the frontend code will then be inside your Encore app. If you go this route with Encore.ts, you will most likely need two different `tsconfig.json` files, one for the frontend and one for the backend. **Related example:** [encoredev/examples/ts/react-starter](https://github.com/encoredev/examples/tree/main/ts/react-starter) React + Encore Starter, frontend code inside the Encore app. Create it locally with `encore app create --example=ts/react-starter`. --- Source: https://encore.dev/docs/dev-dash Section: Building # Local Development Dashboard > Built-in tools for simplicity and productivity Encore provides an efficient local development workflow that automatically provisions [local infrastructure](https://encore.dev/docs/platform/infrastructure/infra#local-development) and supports automated testing with dedicated test infrastructure. The local environment also comes with a built-in Local Development Dashboard to simplify development and improve productivity. It has several features to help you design, develop, and debug your application: * [Service Catalog](https://encore.dev/docs/service-catalog) with Automatic API Documentation * API Explorer to call your APIs * [Distributed Tracing](https://encore.dev/docs/tracing) for simple and powerful debugging * [Architecture diagrams](https://encore.dev/docs/architecture) for visualizing your microservices architecture All these features update in real-time as you make changes to your application. To access the dashboard, start your Encore application with `encore run` and it will open automatically. You can also follow the link in your terminal: ```bash $ encore run API Base URL: http://localhost:4000 Dev Dashboard URL: http://localhost:9400/hello-world-cgu2 ``` --- Source: https://encore.dev/docs/running-scripts Section: Building # Running Scripts > Run scripts with your application's infrastructure and runtime configured and initialized In local development, you may need to run scripts or commands, such as seeding a database with initial data. For that to work the database needs to be started, and the Encore runtime needs to be configured and initialized. ## Using `encore exec` The `encore exec` command allows you to execute custom commands while leveraging Encore's infrastructure setup. This is particularly useful for tasks like database seeding, running scripts, or other one-off commands that require the app's environment to be initialized. ### How it works The `encore exec` command initializes the required infrastructure for your Encore app and executes the specified command. This ensures that your commands run in the correct context with all dependencies properly configured. ### Example: Database Seeding In this example, `npx tsx ./seed.ts` runs a TypeScript script (`seed.ts`) to populate the database with initial data ```bash encore exec -- npx tsx ./seed.ts ``` Here’s what happens: 1. Encore initializes the app infrastructure. 2. The `npx tsx ./seed.ts` command is executed in the context of the initialized app. ### General Syntax ```bash encore exec -- ``` Substitute `` with the specific command you wish to run. ### Use Cases - **Database Seeding**: Populate your database with initial data using a script. - **Client Generation**: Generate a client for interacting with an external dependency. - **Custom Scripts**: Run any script that depends on the app's initialized environment. ### Notes - Ensure that the command you provide is executable in your environment. - Use `--` to separate `encore exec` options from the command you want to run. --- Source: https://encore.dev/docs/infra-namespaces Section: Building # Infrastructure Namespaces > Task switching made easy Encore's CLI allows you to create and switch between multiple, independent *infrastructure namespaces*. Infrastructure namespaces are isolated from each other, and each namespace contains its own independent data. This makes it trivial to switch tasks, confident your old state and data will be waiting for you when you return. If you've ever worked on a new feature that involves making changes to the database schema, only to context switch to reviewing a Pull Request and had to reset your database, you know the feeling. With Encore's infrastructure namespaces, this is a problem of the past. Run `encore namespace switch --create pr:123` (or `encore ns switch -c pr:123` for short) to create and switch to a new namespace. The next `encore run` will run in the new namespace, with a completely fresh database. When you're done, run `encore namespace switch -` to switch back to your previous namespace. ## Usage Below are the commands for working with namespaces. Note that you can use `encore ns` as a short form for `encore namespace`. ```shell # List your namespaces (* indicates the current namespace) $ encore namespace list # Create a new namespace $ encore namespace create my-ns # Switch to a namespace $ encore namespace switch my-ns # Switch to a namespace, creating it if it doesn't exist $ encore namespace switch --create my-ns # Switch to the previous namespace $ encore namespace switch - # Delete a namespace (and all associated data) $ encore namespace delete my-ns ``` Most other Encore commands that interact or use infrastructure take an optional `--namespace` (`-n` for short) that overrides the current namespace. If left unspecified, the current namespace is used. For example: ```shell # Run the app using the "my-ns" namespace $ encore run --namespace my-ns # Open a database shell to the "my-ns" namespace $ encore db shell DATABASE_NAME --namespace my-ns # Reset all databases within the "my-ns" namespace $ encore db reset --all --namespace my-ns ``` --- Source: https://encore.dev/docs/encore-toolbar Section: Building # Encore Toolbar > Inspect API requests and traces from your frontend The Encore Toolbar is a lightweight, drop-in script that adds a floating developer panel to your frontend application. It automatically intercepts all `fetch()` and `XMLHttpRequest` calls, captures trace IDs from Encore's response headers, and lets you jump directly to the corresponding trace in the [Development Dashboard](https://encore.dev/docs/dev-dash) or [the Encore platform](https://encore.dev/docs/platform/introduction). This is useful when you're building a frontend that talks to an Encore backend and want visibility into what's happening on the backend without switching to a separate tool. ## Installation Add a single ` ``` The toolbar automatically detects your App ID and environment by calling the `/__encore/healthz` endpoint on each request's origin. If auto-detection doesn't work (for example, if have a proxy between your frontend and backend and `/__encore/healthz` is not exposed), you can pass them explicitly: ```html ``` | Parameter | Description | | --------- | ----------- | | `appId` | Your Encore app slug (the one in your `app.encore.dev` URL). | | `envName` | The environment name, e.g. `staging`, or `production`. | You can also set or change these values later in the toolbar's Settings panel. > **Note** > > The script patches `window.fetch` and `XMLHttpRequest` at parse time. It must be loaded **before** your application code and **without** `async` or `defer` attributes. Some HTTP libraries (like Axios) store a reference to `fetch` when they initialize, so if the library loads before the toolbar script, those requests won't be intercepted. Place the script tag as early as possible in your ``. ## How it works When your frontend makes a request to an Encore backend, the backend includes an `x-encore-trace-id` header in the response. The toolbar reads this header and records the request along with its trace ID. Only requests that include this header appear in the toolbar. For each captured request, the toolbar shows: - **Method and URL** - the HTTP method and full URL of the request. - **Status code** - the response status. - **Request and response bodies** - captured automatically. - **Query parameters and cookies** - parsed from the request URL and `document.cookie`. - **Trace link** - a direct link to the trace in the Development Dashboard (local) or the Encore platform (deployed environments). - **Backend logs** - when running locally, the toolbar connects to the local Encore daemon and displays backend log output for the selected trace. ## Troubleshooting ### Requests are not being intercepted The toolbar only captures requests that return an `x-encore-trace-id` response header. If your requests aren't showing up: 1. **Verify the header is present.** Open your browser's Network tab, select a request to your Encore backend, and check the response headers for `x-encore-trace-id`. If the header is missing, the request is not going through Encore's request handling (for example, it might be hitting a non-Encore server or a reverse proxy that strips headers). 2. **Make sure the script loads before your app.** The toolbar patches `fetch` and `XMLHttpRequest` at parse time. If your application code runs before the script is loaded, those early requests won't be captured. Move the ` ``` If you don't want to hardcode the environment name, you can add a raw endpoint that redirects to the toolbar script with the correct parameters: **Encore.ts:** ```ts import { api } from "encore.dev/api"; import { appMeta } from "encore.dev"; export const toolbar = api.raw( { method: "GET", expose: true, path: "/encore-toolbar.js" }, async (req, resp) => { const appId = appMeta().appId; const envName = appMeta().environment.name; const url = `https://encore.dev/encore-toolbar.js?appId=${encodeURIComponent(appId)}&envName=${encodeURIComponent(envName)}`; resp.writeHead(302, { Location: url }); resp.end(); }, ); ``` **Encore.go:** ```go package toolbar import ( "fmt" "net/http" "net/url" "encore.dev" ) //encore:api public raw path=/encore-toolbar.js func Toolbar(w http.ResponseWriter, req *http.Request) { meta := encore.Meta() target := fmt.Sprintf( "https://encore.dev/encore-toolbar.js?appId=%s&envName=%s", url.QueryEscape(meta.AppID), url.QueryEscape(meta.Environment.Name), ) http.Redirect(w, req, target, http.StatusFound) } ``` Then point your script tag at your own backend instead: ```html ``` 2. **Check that healthz is reachable.** The toolbar auto-detects App ID and environment by calling `/__encore/healthz` on the request's origin. If you have a reverse proxy or API gateway between your frontend and the Encore backend, the `/__encore/healthz` endpoint may not be exposed through it. In that case, either configure your proxy to forward `/__encore/healthz` to the backend, or pass `appId` and `envName` explicitly on the script tag. 3. **Set values in the toolbar.** Open the toolbar's Settings panel and fill in the App ID and Environment fields manually. ### Backend logs are not loading Backend log streaming only works when running locally. The toolbar connects to the local Encore daemon over WebSocket at `localhost:9400` to fetch logs for a given trace. 1. **Make sure your app is running.** Backend logs require `encore run` to be active. 2. **Check the environment.** Log streaming is only available for the `local` environment. In deployed environments, use the trace link to view logs in [the Encore platform](https://app.encore.dev). 3. **Verify the App ID is set.** The toolbar needs a valid App ID to request logs from the daemon. If `/__encore/healthz` is not reachable then set the App ID via the script tag or in the toolbars Settings. --- Source: https://encore.dev/docs/mcp Section: Building # Local MCP Server > The Model Context Protocol (MCP) exposes tools that provide application context to LLMs. Encore provides an MCP server that implements the [Model Context Protocol](https://modelcontextprotocol.io/introduction), an open standard that enables large language models (LLMs) to access contextual information about your application. Think of MCP as a standardized interface—like a "USB-C port for AI applications"—that connects your Encore app's data and functionality to any LLM that supports the protocol. You can connect to Encore's MCP server from any MCP host (such as Claude Desktop, IDEs, or other AI tools) using either Server-Sent Events (SSE) or stdio transport. To set up this connection, simply run: ```bash cd my-encore-app encore mcp start MCP Service is running! MCP SSE URL: http://localhost:9900/sse?app=your-app-id MCP stdio Command: encore mcp run --app=your-app-id ``` Copy the appropriate URL or command to your MCP host's configuration, and you're ready to give your AI assistants rich context about your application. ## Example: Integrating with Cursor [Cursor](https://cursor.com) is one of the most popular AI powered IDE's, and it's simple to use Encore's MCP server together with Cursor. In order to add the Encore MCP server to Cursor, the fastest way is via the button below (make sure to update `your-app-id` in the configuration to the app ID that `encore mcp start` prints. An app that isn't linked to Encore Cloud has no ID in `encore.app`, so use the printed one). [![Add encore-local MCP server to Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=encore-local&config=eyJjb21tYW5kIjoiZW5jb3JlIG1jcCBydW4gLS1hcHA9eW91ci1hcHAtaWQifQ%3D%3D) If you prefer to configure it manually, create the file `.cursor/mcp.json` with the following settings: ```json -- .cursor/mcp.json -- { "mcpServers": { "encore-local": { "command": "encore", "args": ["mcp", "run", "--app=your-app-id"] } } } ``` Learn more in [Cursor's MCP docs](https://docs.cursor.com/context/model-context-protocol) Now when using Cursor's Agent mode, you can ask it to do advanced actions, such as: "Add an endpoint that publishes to a pub/sub topic, call it and verify that the publish is in the traces" ## Command Reference #### Start Starts an SSE-based MCP server and displays connection information. ```shell $ encore mcp start [--app=] ``` #### Run Establishes an stdio-based MCP session. This command is typically used by MCP hosts to communicate with the server through standard input/output streams. ```shell $ encore mcp run [--app=] ``` ## Exposed Tools Encore's MCP server exposes the following tools that provide AI models with detailed context about your application. These tools enable LLMs to understand your application's structure, retrieve relevant information, and take actions within your system. #### Database Tools - **get_databases**: Retrieve metadata about all SQL databases defined in the application, including their schema, tables, and relationships. - **query_database**: Execute SQL queries against one or more databases in the application. #### API Tools - **call_endpoint**: Make HTTP requests to any API endpoint in the application. - **get_services**: Retrieve comprehensive information about all services and their endpoints in the application. - **get_middleware**: Retrieve detailed information about all middleware components in the application. - **get_auth_handlers**: Retrieve information about all authentication handlers in the application. #### Trace Tools - **get_traces**: Search and retrieve a list of root request traces from the application, with filtering by service, endpoint, Pub/Sub topic/subscription, error status, time range, duration, and parent trace. - **get_trace_spans**: Retrieve detailed information about one or more traces, including all spans, timing information, and associated metadata. #### Source Code Tools - **get_metadata**: Retrieve the complete application metadata, including service definitions, database schemas, API endpoints, and other infrastructure components. #### PubSub Tools - **get_pubsub**: Retrieve detailed information about all PubSub topics and their subscriptions in the application. #### Storage Tools - **get_storage_buckets**: Retrieve comprehensive information about all storage buckets in the application. - **get_objects**: List and retrieve metadata about objects stored in one or more storage buckets. #### Cache Tools - **get_cache_keyspaces**: Retrieve comprehensive information about all cache keyspaces in the application. #### Metrics Tools - **get_metrics**: Retrieve comprehensive information about all metrics defined in the application. #### Cron Tools - **get_cronjobs**: Retrieve detailed information about all scheduled cron jobs in the application. #### Secret Tools - **get_secrets**: Retrieve metadata about all secrets used in the application. #### Documentation Tools - **search_docs**: Search the Encore documentation using Algolia's search engine. - **get_docs**: Retrieve the full content of specific documentation pages. --- Source: https://encore.dev/docs/cors Section: Building # CORS > Configure CORS (Cross-Origin Resource Sharing) for your Encore application CORS is a web security concept that defines which website origins are allowed to access your API. A deep-dive into CORS is out of scope for this documentation, but [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) provides a good overview. In short, CORS affects requests made by browsers to resources hosted on other origins (a combination of the scheme, domain, and port). ## Configuring CORS Encore provides a default CORS configuration that is suitable for many APIs. You can override these settings by specifying the `global_cors` key in the `encore.app` file, which has the following structure: ```cue -- encore.app -- "global_cors": { // debug enables CORS debug logging. "debug": true | false, // allow_headers allows an app to specify additional headers that should be // accepted by the app. // // If the list contains "*", then all headers are allowed. "allow_headers": [...string], // expose_headers allows an app to specify additional headers that should be // exposed from the app, beyond the default set always recognized by Encore. // // If the list contains "*", then all headers are exposed. "expose_headers": [...string], // allow_origins_without_credentials specifies the allowed origins for requests // that don't include credentials. If nil it defaults to allowing all domains // (equivalent to ["*"]). "allow_origins_without_credentials": [...string], // allow_origins_with_credentials specifies the allowed origins for requests // that include credentials. If a request is made from an Origin in this list // Encore responds with Access-Control-Allow-Origin: . // // The URLs in this list may include wildcards (e.g. "https://*.example.com" // or "https://*-myapp.example.com"). "allow_origins_with_credentials": [...string], } ``` ## Allowed origins The main CORS configuration is the list of allowed origins, meaning which websites are allowed to access your API (via browsers). For this purpose, CORS makes a distinction between requests that contain authentication information (cookies, HTTP authentication, or client certificates) and those that do not. CORS applies stricter rules to authenticated requests. By default, Encore allows unauthenticated requests from all origins but disallows requests that do include authorization information from other origins. This is a good default for many APIs. This can be changed by setting the `allow_origins_without_credentials` key (see above). For convenience Encore also allows all origins when developing locally. For security reasons it's necessary to explicitly specify which origins are allowed to make authenticated requests. This is done by setting the `allow_origins_with_credentials` key (see above). ## Allowed headers and exposed headers CORS also lets you specify which headers are allowed to be sent by the client ("allowed headers"), and which headers are exposed to scripts running in the browser ("exposed headers"). Encore automatically configures headers by parsing your program using static analysis. If your API defines a request or response type that contains a header field, Encore automatically adds the header to the list of exposed and allowed headers in request types respectively. To add additional headers to these lists, you can set the `allow_headers` and `expose_headers` keys (see above). This can be useful when your application relies on custom headers in e.g. raw endpoints that aren't seen by Encore's static analysis. --- Source: https://encore.dev/docs/postgres-extensions Section: Building # PostgreSQL Extensions > Pre-installed extensions Encore uses the [encoredotdev/postgres](https://github.com/encoredev/postgres-image) docker image for local development, CI/CD, and for databases hosted on Encore Cloud. The docker image ships with the following PostgreSQL extensions pre-installed and available for use (via `CREATE EXTENSION`): | Extension | Version | Description | | ------------------------------ | ------- | ------------------------------------------------------------------------------------------------------------------- | | refint | 1.0 | functions for implementing referential integrity (obsolete) | | pg_buffercache | 1.3 | examine the shared buffer cache | | pg_freespacemap | 1.2 | examine the free space map (FSM) | | plpgsql | 1.0 | PL/pgSQL procedural language | | citext | 1.6 | data type for case-insensitive character strings | | adminpack | 2.1 | administrative functions for PostgreSQL | | moddatetime | 1.0 | functions for tracking last modification time | | amcheck | 1.3 | functions for verifying relation integrity | | seg | 1.4 | data type for representing line segments or floating-point intervals | | pg_stat_statements | 1.10 | track planning and execution statistics of all SQL statements executed | | pg_trgm | 1.6 | text similarity measurement and index searching based on trigrams | | isn | 1.2 | data types for international product numbering standards | | btree_gist | 1.7 | support for indexing common datatypes in GiST | | intarray | 1.5 | functions, operators, and index support for 1-D arrays of integers | | pg_surgery | 1.0 | extension to perform surgery on a damaged relation | | uuid-ossp | 1.1 | generate universally unique identifiers (UUIDs) | | insert_username | 1.0 | functions for tracking who changed a table | | bloom | 1.0 | bloom access method - signature file based index | | pgcrypto | 1.3 | cryptographic functions | | dblink | 1.2 | connect to other PostgreSQL databases from within a database | | tsm_system_rows | 1.0 | TABLESAMPLE method which accepts number of rows as a limit | | pg_prewarm | 1.2 | prewarm relation data | | old_snapshot | 1.0 | utilities in support of old_snapshot_threshold | | pageinspect | 1.11 | inspect the contents of database pages at a low level | | intagg | 1.1 | integer aggregator and enumerator (obsolete) | | pg_visibility | 1.2 | examine the visibility map (VM) and page-level visibility info | | cube | 1.5 | data type for multidimensional cubes | | tablefunc | 1.0 | functions that manipulate whole tables, including crosstab | | xml2 | 1.1 | XPath querying and XSLT | | fuzzystrmatch | 1.1 | determine similarities and distance between strings | | pg_walinspect | 1.0 | functions to inspect contents of PostgreSQL Write-Ahead Log | | btree_gin | 1.3 | support for indexing common datatypes in GIN | | sslinfo | 1.2 | information about SSL certificates | | tcn | 1.0 | Triggered change notifications | | hstore | 1.8 | data type for storing sets of (key, value) pairs | | dict_int | 1.0 | text search dictionary template for integers | | earthdistance | 1.1 | calculate great-circle distances on the surface of the Earth | | file_fdw | 1.0 | foreign-data wrapper for flat file access | | autoinc | 1.0 | functions for autoincrementing fields | | ltree | 1.2 | data type for hierarchical tree-like structures | | unaccent | 1.1 | text search dictionary that removes accents | | pgrowlocks | 1.2 | show row-level locking information | | tsm_system_time | 1.0 | TABLESAMPLE method which accepts time in milliseconds as a limit | | dict_xsyn | 1.0 | text search dictionary template for extended synonym processing | | pgstattuple | 1.5 | show tuple-level statistics | | postgres_fdw | 1.1 | foreign-data wrapper for remote PostgreSQL servers | | lo | 1.1 | Large Object maintenance | | postgis_sfcgal-3 | 3.4.2 | PostGIS SFCGAL functions | | address_standardizer_data_us-3 | 3.4.2 | Address Standardizer US dataset example | | address_standardizer-3 | 3.4.2 | Used to parse an address into constituent elements. Generally used to support geocoding address normalization step. | | postgis_topology-3 | 3.4.2 | PostGIS topology spatial types and functions | | postgis-3 | 3.4.2 | PostGIS geometry and geography spatial types and functions | | postgis_raster-3 | 3.4.2 | PostGIS raster types and functions | | postgis_tiger_geocoder-3 | 3.4.2 | PostGIS tiger geocoder and reverse geocoder | | vector | 0.7.0 | vector data type and ivfflat and hnsw access methods | | postgis | 3.4.2 | PostGIS geometry and geography spatial types and functions | | address_standardizer | 3.4.2 | Used to parse an address into constituent elements. Generally used to support geocoding address normalization step. | | postgis_topology | 3.4.2 | PostGIS topology spatial types and functions | | postgis_tiger_geocoder | 3.4.2 | PostGIS tiger geocoder and reverse geocoder | | address_standardizer_data_us | 3.4.2 | Address Standardizer US dataset example | | postgis_sfcgal | 3.4.2 | PostGIS SFCGAL functions | | postgis_raster | 3.4.2 | PostGIS raster types and functions | --- Source: https://encore.dev/docs/env-vars Section: Building # Environment Variables > Configure your development environment Encore works out of the box without configuration, but provides several environment variables for advanced use cases such as debugging, testing, or adapting Encore to specific workflow requirements. ## Daemon & Development Dashboard These variables control how the Encore daemon operates and where it exposes its services. ### ENCORE_DAEMON_LOG_PATH Controls the location of the Encore daemon log file. **Default:** `/encore/daemon.log` **Example:** ```bash export ENCORE_DAEMON_LOG_PATH=/var/log/encore/daemon.log ``` ### ENCORE_DEVDASH_LISTEN_ADDR Overrides the listen address for the local development dashboard. **Default:** Automatically assigned by the daemon **Format:** Network address (e.g., `localhost:9400`) **Example:** ```bash export ENCORE_DEVDASH_LISTEN_ADDR=localhost:8080 encore run ``` ### ENCORE_MCPSSE_LISTEN_ADDR Overrides the listen address for the MCP SSE (Model Context Protocol Server-Sent Events) endpoint. **Default:** Automatically assigned by the daemon **Format:** Network address **Example:** ```bash export ENCORE_MCPSSE_LISTEN_ADDR=localhost:9401 ``` ### ENCORE_OBJECTSTORAGE_LISTEN_ADDR Overrides the listen address for the object storage service endpoint. **Default:** Automatically assigned by the daemon **Format:** Network address **Example:** ```bash export ENCORE_OBJECTSTORAGE_LISTEN_ADDR=localhost:9402 ``` ## Logging Configuration ### ENCORE_RUNTIME_LOG (Encore.ts only) Sets the log level for Encore's internal runtime operations (written in Rust). **Default:** `debug` (automatically set to `error` during `encore run`) **Valid values:** `trace`, `debug`, `info`, `warn`, `error` **Example:** ```bash # See detailed runtime logs export ENCORE_RUNTIME_LOG=trace encore run ``` > **Note** > > If `RUST_LOG` is set, it takes precedence over `ENCORE_RUNTIME_LOG`. The runtime log controls logging for internal Encore modules. ### ENCORE_LOG Sets the log level for your application code. **Encore.ts:** **Default:** `Trace` (log everything) **Valid values:** `Off`, `Error`, `Warn`, `Info`, `Debug`, `Trace` ```typescript import log from "encore.dev/log"; log.info("This message respects ENCORE_LOG level"); ``` ```bash # Only show errors and warnings export ENCORE_LOG=Warn encore run ``` **Encore.go:** Especially useful when running `encore test`, where the `--level` flag is not available. **Default:** `trace` (log everything) **Valid values:** `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`, `disabled` ```bash # Only show warnings and above during tests ENCORE_LOG=warn encore test ./... ``` ```bash # Show debug logs during local development ENCORE_LOG=debug encore run ``` > **Note** > > `ENCORE_LOG` takes precedence over the log level set via the `--level` flag or the `log_level` field in `encore.app`. This makes it useful as a per-invocation override. ### ENCORE_NOLOG Disables all logging when set to any non-empty value. **Default:** Not set **Example:** ```bash # Disable all logs export ENCORE_NOLOG=1 encore run ``` ## Advanced Development These variables are primarily useful for advanced development scenarios, such as contributing to Encore itself or using custom builds. ### ENCORE_RUNTIMES_PATH Specifies the path to the Encore runtimes directory. **Default:** Auto-detected relative to the Encore installation (`/runtimes`) **Example:** ```bash export ENCORE_RUNTIMES_PATH=/path/to/custom/runtimes ``` ### ENCORE_RUNTIME_LIB (Encore.ts only) Specifies the path to the native Node.js runtime library used by TypeScript applications. **Default:** `/js/encore-runtime.node` **Example:** ```bash export ENCORE_RUNTIME_LIB=/path/to/custom/encore-runtime.node ``` ### ENCORE_TSPARSER_PATH (Encore.ts only) Specifies the path to the TypeScript parser binary. **Default:** Auto-detected from `encore` binary location or system `PATH` **Example:** ```bash export ENCORE_TSPARSER_PATH=/path/to/custom/tsparser-encore ``` ### ENCORE_GOROOT (Encore.go only) Specifies the path to the custom Encore Go runtime. **Default:** Auto-detected relative to the Encore installation (`/encore-go`) **Example:** ```bash export ENCORE_GOROOT=/path/to/custom/encore-go ``` > **Note** > > For most users, these paths are automatically detected and don't need to be set. They are primarily useful when contributing to Encore or testing custom builds. ## Debugging ### ENCORE_API_INCLUDE_INTERNAL_MESSAGE Controls whether internal error messages are included in API error responses. **Default:** automatically set to `1` during local development with `encore run` **Format:** Any non-empty, non-"0" value is considered `true` **Example:** ```bash # Manually enable for debugging export ENCORE_API_INCLUDE_INTERNAL_MESSAGE=1 ``` ### RUST_LOG (Encore.ts only) Controls Rust-level logging for the Encore runtime. This provides more granular control than `ENCORE_RUNTIME_LOG`. **Default:** Not set **Format:** Standard Rust `env_logger` format (see [env_logger documentation](https://docs.rs/env_logger)) **Example:** ```bash # Enable info logs for all modules in the runtime export RUST_LOG=info encore run ``` > **Note** > > `RUST_LOG` takes precedence over `ENCORE_RUNTIME_LOG`. Use `RUST_LOG` for fine-grained control over specific runtime modules. --- Source: https://encore.dev/docs/ts Section: Building (Encore.ts) # Encore.ts > An open source infrastructure SDK for TypeScript Encore.ts is an open source infrastructure SDK that lets you declare the cloud infrastructure your application needs, such as databases, Pub/Sub topics, object storage, caches, cron jobs, and secrets, directly in your TypeScript code. Instead of stitching together Terraform, YAML, and cloud consoles to wire up these resources, you declare them as type-safe objects and use them through their SDK methods. Encore then takes care of running the matching infrastructure locally during development, and (with [the Encore platform](https://encore.dev/docs/platform/introduction)) provisioning the real cloud resources in your own AWS or GCP account when you deploy. **New to Encore?** The Quickstart guide builds and runs a TypeScript application with two services, a database, and Pub/Sub in a few minutes. - [Start the Quickstart guide](https://encore.dev/docs/quick-start) ## What you get with Encore.ts - **Infrastructure as TypeScript**: Declare [SQL databases](https://encore.dev/docs/ts/primitives/databases), [Pub/Sub topics and subscriptions](https://encore.dev/docs/ts/primitives/pubsub), [object storage buckets](https://encore.dev/docs/ts/primitives/object-storage), [caches](https://encore.dev/docs/ts/primitives/caching), [cron jobs](https://encore.dev/docs/ts/primitives/cron-jobs), and [secrets](https://encore.dev/docs/ts/primitives/secrets) as objects in your code. Encore understands how these resources are used and generates the runtime wiring for you. - **Local infrastructure, automatically**: Run `encore run` and Encore boots a local environment that mirrors production: real Postgres, a local Pub/Sub broker, local object storage, and so on. No Docker Compose files to maintain. - **Type-safe service-to-service calls**: Define APIs as plain TypeScript functions. Calling another service is a normal function call; Encore handles serialization, routing, and validation at the boundary. - **Built-in observability**: A [local development dashboard](https://encore.dev/docs/dev-dash) with distributed [tracing](https://encore.dev/docs/tracing), logs, and a database explorer, so you can see what your application is doing without bolting on extra tools. - **AI-ready by design**: Because infrastructure is declared in code, AI coding assistants can understand and modify your full stack. Encore also ships [AI instructions](https://encore.dev/docs/ai-integration) and an [MCP server](https://encore.dev/docs/mcp) for editor integrations. ## The infrastructure primitives Each primitive is a typed object you declare in the same file as the code that uses it. Together they cover the infrastructure most backend applications run on. Anything outside the set connects as an external dependency. **SQL Database** ```ts import { SQLDatabase } from "encore.dev/storage/sqldb"; const db = new SQLDatabase("orders", { migrations: "./migrations" }); ``` **Pub/Sub** ```ts import { Topic } from "encore.dev/pubsub"; export const orderEvents = new Topic("order-events", { deliveryGuarantee: "at-least-once", }); ``` **Object Storage** ```ts import { Bucket } from "encore.dev/storage/objects"; export const invoices = new Bucket("invoices"); ``` **Caching** ```ts import { CacheCluster } from "encore.dev/storage/cache"; const cluster = new CacheCluster("orders-cache", { evictionPolicy: "allkeys-lru", }); ``` **Cron Jobs** ```ts import { CronJob } from "encore.dev/cron"; const _ = new CronJob("send-invoices", { title: "Send pending invoices", every: "2h", endpoint: sendInvoices, }); ``` **Secrets** ```ts import { secret } from "encore.dev/config"; const stripeKey = secret("StripeAPIKey"); ``` Declarations describe the infrastructure the application needs, and services and APIs are declared the same way. Provisioning settings are configured separately. Encore reads all of it into the [application model](https://encore.dev/docs/understanding-encore), which drives provisioning, IAM and code generation. Declarations are environment-independent, so the same application code runs in local, preview, and production environments without environment-specific checks. See [Development workflow](https://encore.dev/docs/platform/workflow) for the process from `encore run` through per-PR preview environments and production. ## Local and cloud implementations `encore run` starts a local implementation of each one, and in cloud environments [the Encore platform](https://encore.dev/docs/platform/introduction) provisions the managed equivalent in your own AWS or GCP account. You can also provision the infrastructure yourself and point Encore at it with an [infra config file](https://encore.dev/docs/self-host/configure-infra). | Primitive | Locally | AWS | GCP | |---|---|---|---| | [SQL Database](https://encore.dev/docs/ts/primitives/databases) | Postgres in Docker | RDS | Cloud SQL | | [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub) | in-memory | SNS + SQS | Cloud Pub/Sub | | [Object Storage](https://encore.dev/docs/ts/primitives/object-storage) | local filesystem | S3 | Cloud Storage | | [Caching](https://encore.dev/docs/ts/primitives/caching) | in-memory Redis | ElastiCache | Memorystore | | [Cron Jobs](https://encore.dev/docs/ts/primitives/cron-jobs) | not triggered | Encore platform Managed | Encore platform Managed | | [Secrets](https://encore.dev/docs/ts/primitives/secrets) | set per environment | Secrets Manager | Secret Manager | Cron jobs are not triggered automatically in local or preview environments. You can invoke their endpoints from the [development dashboard](https://encore.dev/docs/dev-dash). Compute is chosen per environment rather than declared in code: Cloud Run or GKE on GCP, Fargate or EKS on AWS. [Infrastructure on AWS and GCP](https://encore.dev/docs/platform/infrastructure/infra) also covers networking. ## Services and APIs Services and APIs are included in the application model, where Encore validates their definitions and relationships. - **[App Structure](https://encore.dev/docs/app-structure)**: how an application is laid out, and how services fit together in a monorepo. - **[Services](https://encore.dev/docs/ts/primitives/services)**: group related APIs and infrastructure into independently deployable services. - **[Defining APIs](https://encore.dev/docs/ts/primitives/defining-apis)**: expose typed endpoints from a service. Encore handles request validation, routing and client generation. - **[API Calls](https://encore.dev/docs/ts/primitives/api-calls)**: call another service's API as a regular typed function, wired up in-process locally and over the network in production. ### Request and response data - **[Types](https://encore.dev/docs/ts/primitives/types)**: how types map onto headers, query parameters and path parameters. - **[Validation](https://encore.dev/docs/ts/primitives/validation)**: incoming requests are checked against the declared schema before your handler runs. - **[Errors](https://encore.dev/docs/ts/primitives/errors)**: return structured errors with codes that map to HTTP statuses. - **[Cookies](https://encore.dev/docs/ts/primitives/cookies)**: typed cookie handling. ### Other API styles - **[Raw Endpoints](https://encore.dev/docs/ts/primitives/raw-endpoints)**: access the underlying request and response objects. - **[Streaming APIs](https://encore.dev/docs/ts/primitives/streaming-apis)**: streams in either direction over WebSockets. - **[GraphQL](https://encore.dev/docs/ts/primitives/graphql)**: serve a GraphQL schema from a service. - **[Static Assets](https://encore.dev/docs/ts/primitives/static-assets)**: serve files from a directory. ## From local to production Because the SDK is the source of truth for your infrastructure, the same model runs locally, in per-PR preview environments, and in production. `encore run` boots the whole system on your laptop with real Postgres, real Pub/Sub semantics, and real tracing. Opening a pull request spins up a [preview environment](https://encore.dev/docs/platform/deploy/preview-environments) with the same infrastructure model and (optionally) a [database branched from a seed environment](https://encore.dev/docs/platform/infrastructure/neon). This is the loop that makes Encore.ts particularly effective with AI coding agents: every change can be validated end-to-end against real infrastructure, locally and in the cloud, instead of waiting for a production deploy to find out if it works. For more, see the [Development Workflow](https://encore.dev/docs/platform/workflow) page. Encore.ts is fully open source and you are never locked into a particular deployment path. There are two supported ways to run an Encore.ts application in production. **With [the Encore platform](https://encore.dev/docs/platform/introduction)**: the optional managed platform reads the infrastructure you declared in your code and provisions matching resources (RDS, Cloud SQL, SNS+SQS, Pub/Sub, S3, GCS, etc.) in your own AWS or GCP account, then manages deploys, environments, secrets, and IAM on top. You own the cloud account and the infrastructure; the Encore platform is the control plane that drives it from your code. **Self-hosted**: build a standard Docker image with `encore build docker` and run it anywhere. You provision your own infrastructure however you like (Terraform, Pulumi, CloudFormation, the cloud console), and tell the Encore runtime how to reach those resources via an [infrastructure config file](https://encore.dev/docs/self-host/configure-infra). See the [self-hosting guide](https://encore.dev/docs/self-host/build) for the full workflow. ## Primitives and AI agents Primitives expose infrastructure resources as type-safe objects. Deployment-specific settings are configured per environment. This separation helps agents produce valid declarations, with invalid usage reported by the type system or at build time. For example, an agent can declare a database with only a name and a migrations directory: ```ts const db = new SQLDatabase("orders", { migrations: "./migrations" }); ``` An equivalent Terraform configuration may also define the database instance class, storage type, networking, parameter group, and IAM policy. Encore handles these concerns as follows: - Deployment settings such as instance size, networking, and backups are configured per [environment](https://encore.dev/docs/platform/infrastructure/configuration) and remain outside the application code. - Resource names must be string literals, and resource declarations must be defined at module scope. Invalid declarations are reported at build time. See [Understanding Encore](https://encore.dev/docs/understanding-encore) for the complete requirements. - IAM policies are derived from resource usage, limiting permissions to the relationships declared in the application. - Request schemas are derived from TypeScript types, so invalid client usage and handler signatures produce compile-time errors. See [AI integration](https://encore.dev/docs/ai-integration) for editor rules and an MCP server that expose the service graph, schemas, and traces to agents. [Coming from Terraform](https://encore.dev/docs/platform/migration/from-terraform) maps Terraform concepts to Encore and covers using both together. ## Other infrastructure Primitives do not limit the infrastructure your application can use. For services without a primitive, such as a search cluster or data warehouse, provision them separately, connect to them as external dependencies, and store their connection details in [secrets](https://encore.dev/docs/ts/primitives/secrets). Resources you already run can be connected to a declaration rather than provisioned new. Encore imports existing databases, buckets, and topics on [AWS](https://encore.dev/docs/platform/infrastructure/aws) and [GCP](https://encore.dev/docs/platform/infrastructure/gcp), and deploys into an existing [Kubernetes cluster](https://encore.dev/docs/platform/infrastructure/import-kubernetes-cluster). To see exactly what Encore creates in your cloud, see [Infrastructure on AWS and GCP](https://encore.dev/docs/platform/infrastructure/infra). ## Where to go next - New to Encore? Start with the [Quickstart guide](https://encore.dev/docs/quick-start) to build and run your first application. - Want to understand the model first? Read [App Structure](https://encore.dev/docs/app-structure) and the [Services](https://encore.dev/docs/ts/primitives/services) page. - Curious about the local-to-production iteration loop? See the [Development Workflow](https://encore.dev/docs/platform/workflow). - Ready to deploy? See [Deploy with the Encore platform](https://encore.dev/docs/platform/deploy/deploying) or [Build Docker Images](https://encore.dev/docs/self-host/build) for self-hosting. ## Get started **Install the Encore CLI** (pick the line for your OS): ```shell # macOS brew install encoredev/tap/encore # Windows iwr https://encore.dev/install.ps1 | iex # Linux curl -L https://encore.dev/install.sh | bash ``` Then scaffold an app: ```shell encore app create ``` --- Source: https://encore.dev/docs/ts/primitives/services Section: Building (Encore.ts) # Defining Services > Simplifying (micro-)service development Encore.ts makes it simple to build applications with one or many services, without needing to manually handle the typical complexity of developing microservices. ## Defining services To create an Encore service, add a file named `encore.service.ts` in a directory. The file must export a service instance, by calling `new Service`, imported from `encore.dev/service`. For example: ```ts -- encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("my-service"); ``` That's it! Encore will consider this directory and all its subdirectories as part of the service. With multiple services, each service lives in its own directory with its own `encore.service.ts`: ``` /my-app ├── package.json ├── encore.app │ ├── hello // hello service │ ├── encore.service.ts │ └── hello.ts │ └── world // world service ├── encore.service.ts └── world.ts ``` For more on how to structure your application, see the [app structure guide](https://encore.dev/docs/app-structure). --- Source: https://encore.dev/docs/ts/primitives/databases Section: Building (Encore.ts) # Using SQL databases > Provisioning, migrating, querying Encore treats SQL databases as logical resources and natively supports **PostgreSQL** databases. **Related example:** [encoredev/examples/ts/url-shortener](https://github.com/encoredev/examples/tree/main/ts/url-shortener) URL Shortener example that uses a PostgreSQL database. Create it locally with `encore app create --example=ts/url-shortener`. ## Creating a database To create a database, import `encore.dev/storage/sqldb` and call `new SQLDatabase`, assigning the result to a top-level variable. Use a migration file in a directory `migrations` to define the database schema. For example: ```typescript -- todo/todo.ts -- import { SQLDatabase } from "encore.dev/storage/sqldb"; // Create the todo database and assign it to the "db" variable const db = new SQLDatabase("todo", { migrations: "./migrations", }); // Then, query the database using db.query, db.exec, etc. -- todo/migrations/1_create_table.up.sql -- CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL DEFAULT false -- etc... ); ``` As seen above, the `new SQLDatabase()` call takes two parameters: the name of the database, and a configuration object. The configuration object specifies the directory containing the database migration files, which is how you define the database schema. See the [Defining the database schema](#defining-the-database-schema) section below for more details. With this code in place, Encore will automatically create the database using [Docker](https://docker.com) when you run the command `encore run` in your local environment. Make sure Docker is installed and running on your machine before running `encore run`. > **Note** > > If your application is already running when you define a new database, you will need to stop and restart `encore run`. This is necessary for Encore to create the new database using Docker. In cloud environments, Encore automatically injects the appropriate configuration to authenticate and connect to the database, so once the application starts up the database is ready to be used. ## Database Migrations Encore automatically handles `up` migrations, while `down` migrations must be run manually. Each `up` migration runs sequentially, expressing changes in the database schema from the previous migration. ### Naming Conventions **File Name Format:** Migration files must start with a number followed by an underscore (`_`), and must increase sequentially. Each file name must end with `.up.sql`. **Examples:** - `1_first_migration.up.sql` - `2_second_migration.up.sql` - `3_migration_name.up.sql` You can also prefix migration files with leading zeroes for better ordering in the editor (e.g., `0001_migration.up.sql`). ### Defining the Database Schema The first migration typically defines the initial table structure. For instance, a `todo` service might create `todo/migrations/1_create_table.up.sql` with the following content: ```sql -- todo/migrations/1_create_table.up.sql -- CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL DEFAULT false ); ``` ### Migration File Structure Migration files are created in a `migrations` directory within an Encore service. Each file is named `_.up.sql`, where `` is a sequence number for ordering and `` describes the migration. **Example Directory Structure:** ``` /my-app ├── encore.app // ... other top-level project files │ └── todo // todo service    ├── migrations // database migrations (directory)    │ ├── 1_create_table.up.sql // first migration file    │ └── 2_add_field.up.sql // second migration file    ├── todo.ts // todo service code    └── todo.test.ts // tests for todo service ``` ## Using databases Once you have created the database using `const db = new SQLDatabase(...)` you can start querying and inserting data into the database by calling methods on the `db` variable. ### Querying data To query data, use the following methods: - `db.query`: Returns an asynchronous iterator, yielding rows one by one. - `db.queryRow`: Returns a single row, or `null` if no rows are found. - `db.queryAll`: Returns an array of all rows. - `db.rawQuery`: Similar to `db.query`, but takes a raw SQL string and parameters. - `db.rawQueryRow`: Similar to `db.queryRow`, but takes a raw SQL string and parameters. - `db.rawQueryAll`: Similar to `db.queryAll`, but takes a raw SQL string and parameters. Typical usage looks like this: ```ts const allTodos = await db.query`SELECT * FROM todo_item`; for await (const todo of allTodos) { // Process each todo } ``` Or to query a single todo item by id: ```ts async function getTodoTitle(id: number): string | undefined { const row = await db.queryRow`SELECT title FROM todo_item WHERE id = ${id}`; return row?.title; } ``` Or to query using raw SQL and parameters: ```ts async function getTodoTitle(id: number): string | undefined { const row = await db.rawQueryRow("SELECT title FROM todo_item WHERE id = $1", id); return row?.title; } ``` ### Inserting data To insert data, or to make database queries that don't return any rows, use `db.exec` or `db.rawExec`. For example: ```ts await db.exec` INSERT INTO todo_item (title, done) VALUES (${title}, false) `; ``` Or using raw SQL and parameters: ```ts await db.rawExec( "INSERT INTO todo_item (title, done) VALUES ($1, $2)", title, false ); ``` ### Transactions Transactions allow you to group multiple database operations into a single unit of work. If any operation within the transaction fails, the entire transaction is rolled back, ensuring data consistency. The transaction type implements `AsyncDisposable`, which automatically rolls back the transaction if it is not explicitly committed or rolled back. This ensures that no open transactions are left accidentally. Queries must be run on the transaction (`tx`), not the database (`db`), to be part of it. For example: ```ts await using tx = await db.begin(); await tx.exec` INSERT INTO todo_item (title, done) VALUES (${title1}, false) `; await tx.exec` INSERT INTO todo_item (title, done) VALUES (${title2}, false) `; await tx.commit(); ``` ## Connecting to databases It's often useful to be able to connect to the database from outside the backend application. For example for scripts, ad-hoc querying, or dumping data for analysis. Currently Encore does not expose user credentials for databases in the local environment or for environments on Encore Cloud. You can use a connection string to connect instead, see below. If you need to connect external tools like data pipelines or BI platforms, see [Connecting external tools to databases](https://encore.dev/docs/platform/infrastructure/managing-infrastructure#connecting-external-tools-to-databases) for guidance on SSL certificates and network access. ### Using the Encore CLI Encore's CLI comes with built-in support for connecting to databases: * `encore db shell [--env=]` opens a [psql](https://www.postgresql.org/docs/current/app-psql.html) shell to the database named `` in the given environment. Leaving out `--env` defaults to the local development environment. `encore db shell` defaults to read-only permissions. Use `--write`, `--admin` and `--superuser` flags to modify which permissions you connect with. * `encore db conn-uri [--env=]` outputs a connection string for the database named ``. When specifying a cloud environment, the connection string is temporary. Leaving out `--env` defaults to the local development environment. * `encore db proxy [--env=]` sets up a local proxy that forwards any incoming connection to the databases in the specified environment. Leaving out `--env` defaults to the local development environment. See `encore help db` for more information on database management commands. ### Using database user credentials For cloud environments on AWS/GCP you can view database user credentials (created by Encore when provisioning databases) via the Cloud Dashboard: * Open your app in the [Encore platform dashboard](https://app.encore.dev), navigate to the **Infrastructure** page for the appropriate environment, and locate the `USERS` section within the relevant **Database Cluster**. ## Handling migration errors When Encore applies database migrations, there's always a possibility the migrations don't apply cleanly. This can happen for many reasons: - There's a problem with the SQL syntax in the migration - You tried to add a `UNIQUE` constraint but the values in the table aren't actually unique - The existing database schema didn't look like you thought it did, so the database object you tried to change doesn't actually exist - ... and so on If that happens, Encore rolls back the migration. If it happens during a cloud deployment, the deployment is aborted. Once you fix the problem, re-run `encore run` (locally) or push the updated code (in the cloud) to try again. Encore tracks which migrations have been applied in the `schema_migrations` table: ```sql database=# \d schema_migrations Table "public.schema_migrations" Column | Type | Collation | Nullable | Default ---------+---------+-----------+----------+--------- version | bigint | | not null | dirty | boolean | | not null | Indexes: "schema_migrations_pkey" PRIMARY KEY, btree (version) ``` The `version` column tracks which migration was last applied. If you wish to skip a migration or re-run a migration, change the value in this column. For example, to re-run the last migration, run `UPDATE schema_migrations SET version = version - 1;`. *Note that Encore does not use the `dirty` flag by default.* ## The `encore_services` role Encore uses a shared database role named `encore_services` to bridge permissions between database migrations and the services that connect to the database at runtime. This role exists consistently across all environments — local and cloud. - All roles that run migrations are granted `encore_services`. - All roles used by services to connect to the database at runtime are also granted `encore_services`. This means anything owned by `encore_services` — or any permissions granted to it — is automatically accessible to both migration roles and runtime service roles. You can use migration scripts to grant additional permissions to `encore_services`, and they will be inherited by the service roles connecting at runtime. ### Materialized views example Materialized views are a good example of where `encore_services` is useful: they're created during migrations, but need to be refreshed at runtime by the service role. You can bridge this gap by creating a dedicated owner role in a migration, granting it to `encore_services`, and assuming it when creating the view: ```sql -- In a migration file CREATE ROLE matview_owner; GRANT matview_owner TO encore_services; SET ROLE matview_owner; CREATE MATERIALIZED VIEW my_view AS SELECT id, count(*) AS total FROM todo_item GROUP BY id; RESET ROLE; ``` At runtime, the service role can assume `matview_owner` (via `encore_services`) and refresh the view: ```ts await db.exec`SET ROLE matview_owner`; await db.exec`REFRESH MATERIALIZED VIEW my_view`; await db.exec`RESET ROLE`; ``` ### Legacy behavior In earlier versions of Encore, migrations were run using the same database account that services used to connect at runtime. To restore this legacy behavior, set the following environment variable: ```shell ENCOREDEBUG="sqldbrole=legacy" ``` This is provided for backwards compatibility and is not recommended for new applications. ## Using an ORM Encore has all the tools needed to support ORMs and migration frameworks out-of-the-box through named databases and migration files. Writing plain SQL might not work for your use case, or you may not want to use SQL in the first place. ORMs like [Prisma](https://encore.dev/docs/ts/develop/orms/prisma) and [Drizzle](https://encore.dev/docs/ts/develop/orms/drizzle) can be used with Encore by integrating their logic with a system's database. Encore is not restrictive, it uses plain SQL migration files for its migrations. * If your ORM of choice can connect to any database using a standard SQL driver, then it can be used with Encore. * If your migration framework can generate SQL migration files without any modifications, then it can be used with Encore. For more information on using ORMs with Encore, see the [ORMs](https://encore.dev/docs/ts/develop/orms) page. ## Sharing databases between services There are two primary ways of sharing a database between services: - You can define the `SQLDatabase` object in a shared module as an exported variable, and reference this object from every service that needs to access the database. - You can define the `SQLDatabase` object in one service using `new SQLDatabase("name", ...)`, and have other services access it by creating a reference using `SQLDatabase.named("name")`. Both approaches have the same effect, but the latter is more explicit. ## PostgreSQL Extensions Encore uses the [encoredotdev/postgres](https://github.com/encoredev/postgres-image) docker image for local development, CI/CD, and for databases hosted on Encore Cloud. This docker image ships with many popular PostgreSQL extensions pre-installed. In particular, [pgvector](https://github.com/pgvector/pgvector) and [PostGIS](https://postgis.net) are available. See [the full list of available extensions](https://encore.dev/docs/postgres-extensions). ## Troubleshooting When you run your application locally with `encore run`, Encore will provision local databases using Docker. If this fails with a database error, it can often be resolved if you restart the Encore daemon using `encore daemon` and then try `encore run` again. If this does not resolve the issue, here are steps to resolve common errors: **Error: sqldb: unknown database** This error is often caused by a problem with the initial migration file, such as incorrect naming or location. - Verify that you've [created the migration file](#defining-the-database-schema) correctly, then try `encore run` again. **Error: could not connect to the database** When you can't connect to the database in your local environment, there's likely an issue with Docker: - Make sure that you have [Docker](https://docker.com) installed and running, then try `encore run` again. - If this fails, restart the Encore daemon by running `encore daemon`, then try `encore run` again. **Error: Creating PostgreSQL database cluster Failed** This means Encore was not able to create the database. Often this is due to a problem with Docker. - Check if you have permission to access Docker by running `docker images`. - Set the correct permissions with `sudo usermod -aG docker $USER` (Learn more in the [Docker documentation](https://docs.docker.com/engine/install/linux-postinstall/)) - Then log out and log back in so that your group membership is refreshed. **Error: unable to save docker image** This error is often caused by a problem with Docker. - Make sure that you have [Docker](https://docker.com) installed and running. - In Docker, open **Settings > Advanced** and make sure that the setting `Allow the default Docker socket to be used` is checked. - If it still fails, restart the Encore daemon by running `encore daemon`, then try `encore run` again. **Error: unable to add CA to cert pool** This error is commonly caused by the presence of the file `$HOME/.postgresql/root.crt` on the filesystem. When this file is present the PostgreSQL client library will assume the database server has that root certificate, which will cause the above error. - Remove or rename the file, then try `encore run` again. **Resetting databases** If your local database is in a bad state (e.g. due to an incomplete migration or corrupt data), you can reset it by running: ```shell $ encore db reset ``` This drops and recreates the database, re-running all migrations from scratch. Use `--all` to reset all databases at once. --- Source: https://encore.dev/docs/ts/develop/orms Section: Building (Encore.ts) # Using ORMs and Migration Frameworks with Encore.ts Encore provides built-in support for ORMs and migration frameworks by offering named databases and SQL-based migration files. For developers who prefer not to write raw SQL, Encore allows seamless integration with popular ORMs and migration tools. ## Overview Encore’s approach to database management is flexible. It uses standard SQL migration files, allowing integration with ORMs like [Sequelize](https://sequelize.org/) and migration tools like [Atlas](https://atlasgo.io/). - **ORM Compatibility:** If your ORM can connect to a database via a standard SQL driver, it will work with Encore. - **Migration Tool Compatibility:** If your migration tool generates SQL migration files without additional customization, it can be used with Encore. ## Connecting to a Database Encore provides the `SQLDatabase` class, which allows you to create a named database and retrieve its connection string. This connection string can be used by your chosen ORM or migration framework to establish a database connection. Example setup: ```typescript import { SQLDatabase } from "encore.dev/storage/sqldb"; // Initialize a named database with migration directory const SiteDB = new SQLDatabase("siteDB", { migrations: "./migrations", }); // Retrieve the connection string for ORM use const connStr = SiteDB.connectionString; ``` ## Example ORM implementations Here are some guides to using different ORMs with Encore: - [Using Knex.js with Encore](https://encore.dev/docs/ts/develop/orms/knex) - [Using Sequelize with Encore](https://encore.dev/docs/ts/develop/orms/sequelize) - [Using Drizzle with Encore](https://encore.dev/docs/ts/develop/orms/drizzle) - [Using Prisma with Encore](https://encore.dev/docs/ts/develop/orms/prisma) This setup enables Encore to support a wide variety of ORMs and migration frameworks, making database management both flexible and straightforward. --- Source: https://encore.dev/docs/ts/develop/orms/knex Section: Building (Encore.ts) # Using Knex.js with Encore Encore.ts supports integrating [Knex.js](http://knexjs.org/), a SQL query builder for Node.js. To use Knex.js with Encore.ts, start by creating an `SQLDatabase` instance and provide its connection string to Knex.js. ## 1. Setting Up the Database Connection In `site.ts`, initialize the `SQLDatabase` and configure Knex.js: ```typescript -- site.ts -- import { SQLDatabase } from "encore.dev/storage/sqldb"; import knex from "knex"; // Create SQLDatabase instance with migrations configuration const SiteDB = new SQLDatabase("siteDB", { migrations: "./migrations", }); // Initialize Knex with the database connection string const orm = knex({ client: "pg", connection: SiteDB.connectionString, }); // Define the Site interface export interface Site { id: number; url: string; } // Query builder for the "site" table const Sites = () => orm("site"); // Example queries // Query all sites await Sites().select(); // Query a site by id await Sites().where("id", id).first(); // Insert a new site await Sites().insert({ url: params.url }); ``` ## 2. Creating Migrations Currently, Encore does not support JavaScript migration files generated by `knex migrate:make`. Instead, you can create and maintain [migration files](https://encore.dev/docs/ts/primitives/databases#database-migrations) in SQL format. Example migration file to create the `site` table: ```sql -- migrations/1_create_table.up.sql -- CREATE TABLE site ( id SERIAL PRIMARY KEY, url TEXT NOT NULL UNIQUE ); ``` ## 3. Applying Migrations Encore automatically applies migrations when you run your application. You do not need to run `knex migrate:latest` or similar commands manually. **Related example:** [encoredev/examples/ts/knex](https://github.com/encoredev/examples/tree/main/ts/knex) Example implementation showing how to use Knex ORM with Encore.ts Create it locally with `encore app create --example=ts/knex`. --- Source: https://encore.dev/docs/ts/develop/orms/prisma Section: Building (Encore.ts) # Using Prisma with Encore.ts [Prisma](https://prisma.io/) is a modern TypeScript ORM that provides type-safe database access and migrations. With Prisma, you define your database schema in a `schema.prisma` file and use Prisma's CLI to generate SQL migrations and a TypeScript client. This guide explains how to integrate Prisma with Encore.ts, leveraging Encore's built-in database management while using Prisma's powerful ORM features. ## How Prisma works with Encore Encore and Prisma work together seamlessly: - **Prisma** generates the migration files and TypeScript client - **Encore** manages database creation, connections, and applies migrations - You use Encore's `SQLDatabase` to provide connection strings to Prisma The key to this integration is configuring Prisma to use Encore's shadow database for its operations, preventing any conflicts between the two systems. ## Quick Example Here's a complete example of using Prisma with Encore.ts: ```ts -- users/database.ts -- import { SQLDatabase } from "encore.dev/storage/sqldb"; // Define a database named 'users', using the database migrations // in the "./prisma/migrations" folder (generated by Prisma). export const DB = new SQLDatabase("users", { migrations: { path: "./prisma/migrations", source: "prisma", }, }); -- users/prisma/schema.prisma -- generator client { provider = "prisma-client" output = "./generated" previewFeatures = ["queryCompiler", "driverAdapters"] } datasource db { provider = "postgresql" // Connect Prisma CLI to Encore's shadow database // This prevents interference with Encore's migration system url = env("SHADOW_DB_URL") } model User { id Int @id @default(autoincrement()) email String @unique name String createdAt DateTime @default(now()) } -- users/prisma/client.ts -- import { PrismaClient } from "./generated/client"; import { PrismaPg } from "@prisma/adapter-pg"; import { DB } from "../database"; // Create and export the Prisma client instance export const prisma = new PrismaClient({ adapter: new PrismaPg({ connectionString: DB.connectionString }), }); // Re-export types from the generated client export * from "./generated/client"; -- users/api.ts -- import { api } from "encore.dev/api"; import { prisma } from "./prisma/client"; interface CreateUserRequest { email: string; name: string; } // Example API endpoint using Prisma export const createUser = api( { method: "POST", path: "/users", expose: true }, async (req: CreateUserRequest) => { const user = await prisma.user.create({ data: req, }); return user; } ); ``` **Related example:** [encoredev/examples/ts/prisma](https://github.com/encoredev/examples/tree/main/ts/prisma) Complete Prisma + Encore.ts example Create it locally with `encore app create --example=ts/prisma`. ## Step-by-Step Setup ### 1. Install Dependencies First, install Prisma and its required dependencies: ```bash npm install prisma --save-dev npm install @prisma/client @prisma/adapter-pg dotenv --save ``` ### 2. Create Project Structure Create the following directory structure for your service: ``` my-service/ ├── database.ts ├── prisma/ │ ├── schema.prisma │ ├── migrations/ │ ├── generated/ (will be created by Prisma) │ └── client.ts └── api.ts ``` ### 3. Set Up the Database Create `database.ts` to define your Encore database: ```ts -- database.ts -- import { SQLDatabase } from "encore.dev/storage/sqldb"; // Export the database so it can be used in the Prisma client export const DB = new SQLDatabase("myapp", { migrations: { path: "./prisma/migrations", source: "prisma", }, }); ``` Run `encore run` to create the database (make sure the migrations folder has been created first). ### 4. Configure Database Connections Prisma needs to connect to Encore's shadow database for migration operations. The shadow database is a temporary database that Prisma uses to detect schema drift and generate migrations without affecting your main database. Get the connection strings: ```bash # Main database connection (for Prisma Studio) encore db conn-uri myapp # Shadow database connection (for Prisma CLI operations) encore db conn-uri myapp --shadow ``` Create a `.env` file in your project root: ``` # Connection strings for local development DB_URL= SHADOW_DB_URL= ``` ### 5. Create Prisma Configuration Create `prisma.config.ts` in your project root: ```ts -- prisma.config.ts -- import "dotenv/config"; import type { PrismaConfig } from "prisma"; import { PrismaPg } from "@prisma/adapter-pg"; type Env = { DB_URL: string; }; export default { earlyAccess: true, schema: "./my-service/prisma/schema.prisma", studio: { adapter: async (env: Env) => { // Connect Prisma Studio to the main Encore database return new PrismaPg({ connectionString: env.DB_URL }); }, }, } satisfies PrismaConfig; ``` ### 6. Create Your Prisma Schema Create `my-service/prisma/schema.prisma`: ``` generator client { provider = "prisma-client" output = "./generated" previewFeatures = ["queryCompiler", "driverAdapters"] } datasource db { provider = "postgresql" // IMPORTANT: Use shadow database URL for Prisma CLI operations url = env("SHADOW_DB_URL") } // Define your models here model User { id Int @id @default(autoincrement()) email String @unique name String } ``` ### 7. Create the Prisma Client Wrapper Create `my-service/prisma/client.ts`: ```ts -- my-service/prisma/client.ts -- import { PrismaClient } from "./generated/client"; import { PrismaPg } from "@prisma/adapter-pg"; import { DB } from "../database"; // Create and export the Prisma client instance export const prismaClient = new PrismaClient({ adapter: new PrismaPg({ connectionString: DB.connectionString }), }); // Re-export types from the generated client export * from "./generated/client"; ``` ### 8. Generate Initial Migration Generate and apply your first migration: ```bash npx prisma migrate dev --name init encore run ``` This will: 1. Create the `prisma/migrations` directory 2. Generate SQL migration files 3. Generate the Prisma client in `prisma/generated` 4. Apply the migration to your development database ### 9. Use Prisma in Your Code Now you can use Prisma in your API endpoints: ```ts import { api, APIError } from "encore.dev/api"; import { prismaClient, Prisma } from "./prisma/client"; interface CreateUserRequest { email: string; name: string; } export const createUser = api( { method: "POST", path: "/users", expose: true }, async (req: CreateUserRequest) => { try { const user = await prismaClient.user.create({ data: req, }); return user; } catch (error) { if (error instanceof Prisma.PrismaClientKnownRequestError) { if (error.code === "P2002") { throw APIError.alreadyExists("User with this email already exists"); } } throw error; } }, ); export const getUsers = api( { method: "GET", path: "/users", expose: true }, async (): Promise<{ users: { name: string; email: string; id: number }[]; }> => { return { users: await prismaClient.user.findMany() }; }, ); ``` ## Working with Migrations ### Generate New Migrations When you make changes to your `schema.prisma` file: ```bash npx prisma migrate dev --name describe-your-change ``` Example: ```bash npx prisma migrate dev --name add-user-role ``` ### How Migrations Work 1. **Prisma generates** SQL migration files based on your schema changes 2. **Encore applies** these migrations automatically: - Locally when you run `encore run` - In cloud environments during deployment - During tests when using `encore test` ## Deployment ### Generate Client During Build For the Encore platform deployments, add to your `package.json`: ```json -- package.json -- { "scripts": { "postinstall": "npx prisma generate" } } ``` This ensures the Prisma client is generated during deployment. ### Production Considerations - Encore automatically provides database connection strings in production - No environment variables need to be configured for production - Migrations are applied automatically during deployment ## Using Prisma Studio Prisma Studio provides a GUI for your database: ```bash npx prisma studio ``` Opens at `http://localhost:5555` where you can: - Browse and filter data - Create, update, and delete records - Explore relationships ## Troubleshooting ### Connection Issues If Prisma or Prisma Studio can't connect: 1. Verify connection strings in `.env` 2. Restart Encore (`encore run`) --- Source: https://encore.dev/docs/ts/develop/orms/drizzle Section: Building (Encore.ts) # Using Drizzle ORM with Encore Encore.ts supports integrating [Drizzle](https://orm.drizzle.team/), a TypeScript ORM for Node.js and the browser. To use Drizzle with Encore, start by creating a `SQLDatabase` instance and providing the connection string to Drizzle. **Related example:** [encoredev/examples/ts/drizzle](https://github.com/encoredev/examples/tree/main/ts/drizzle) Using Drizzle ORM with Encore.ts Create it locally with `encore app create --example=ts/drizzle`. ## 1. Setting Up the Database Connection In `database.ts`, initialize the `SQLDatabase` and configure Drizzle: ```typescript -- database.ts -- import { api } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { drizzle } from "drizzle-orm/node-postgres"; import { users } from "./schema"; // Create SQLDatabase instance with migrations configuration const db = new SQLDatabase("test", { migrations: { path: "migrations", source: "drizzle", }, }); // Initialize Drizzle ORM with the connection string const orm = drizzle(db.connectionString); // Query all users await orm.select().from(users); ``` ## 2. Configuring Drizzle Create a Drizzle configuration file `drizzle.config.ts` to specify settings like migration output, schema, and database dialect: ```typescript -- drizzle.config.ts -- import 'dotenv/config'; import { defineConfig } from 'drizzle-kit'; export default defineConfig({ out: 'migrations', schema: 'schema.ts', dialect: 'postgresql', }); ``` ## 3. Defining the Database Schema Define your database tables in `schema.ts` using Drizzle's `pg-core` module: ```typescript -- schema.ts -- import * as p from "drizzle-orm/pg-core"; export const users = p.pgTable("users", { id: p.serial().primaryKey(), name: p.text(), email: p.text().unique(), }); ``` ## 4. Generating Migrations Run the following command in the directory containing `drizzle.config.ts` to generate migrations: ```bash drizzle-kit generate ``` ## 5. Applying Migrations Migrations are automatically applied when you run your Encore application, so you don’t need to run `drizzle-kit migrate` or any similar commands manually. --- Source: https://encore.dev/docs/ts/develop/orms/sequelize Section: Building (Encore.ts) # Using Sequelize with Encore.ts Encore.ts supports integrating [Sequelize](https://sequelize.org/), a promise-based Node.js ORM. To set up Sequelize with Encore, start by creating a `SQLDatabase` instance and providing the connection string to Sequelize. ## 1. Setting Up the Database Connection In `database.ts`, initialize the `SQLDatabase` and configure Sequelize: ```typescript -- database.ts -- import { Model, InferAttributes, InferCreationAttributes, CreationOptional, DataTypes, Sequelize, } from "sequelize"; import { SQLDatabase } from "encore.dev/storage/sqldb"; // Create SQLDatabase instance with migrations configuration const DB = new SQLDatabase("encore_sequelize_test", { migrations: "./migrations", }); // Initialize Sequelize with the connection string const sequelize = new Sequelize(DB.connectionString); // Define the User model class User extends Model, InferCreationAttributes> { declare id: CreationOptional; declare name: string; declare surname: string; } // Example usage: Count all users const count = await User.count(); ``` ## 2. Creating Migrations Encore does not currently support JavaScript migration files generated by tools like `sequelize-cli model:generate`. Instead, create and manage your own [migration files](https://encore.dev/docs/ts/primitives/databases#database-migrations) in SQL format. Example migration file for creating the `user` table: ```sql -- migrations/1_create_user.up.sql -- CREATE TABLE "user" ( id SERIAL PRIMARY KEY, name TEXT NOT NULL, surname TEXT NOT NULL ); ``` ## 3. Applying Migrations Migrations are automatically applied when you run your Encore application, so you don’t need to run `sequelize db:migrate` or similar commands manually. --- For more information, see the example on GitHub: **Related example:** [encoredev/examples/ts/sequelize](https://github.com/encoredev/examples/tree/main/ts/sequelize) Using Sequelize ORM with Encore.ts Create it locally with `encore app create --example=ts/sequelize`. --- Source: https://encore.dev/docs/ts/primitives/object-storage Section: Building (Encore.ts) # Object Storage > Simple and scalable storage APIs for files and unstructured data Object Storage is a simple and scalable solution to store files and unstructured data in your backend application. The most common implementation is Amazon S3 ("Simple Storage Service") and its semantics are universally supported by every major cloud provider. Encore.ts provides a cloud-agnostic API for working with Object Storage, allowing you to store and retrieve files with ease. It has support for Amazon S3, Google Cloud Storage, as well as any other S3-compatible implementation (such as DigitalOcean Spaces, MinIO, etc.). Additionally, when you use Encore's Object Storage API you also automatically get: * Automatic tracing and instrumentation of all Object Storage operations * Built-in local development support, storing objects on the local filesystem * Support for integration testing, using a local, in-memory storage backend **Related example:** [encoredev/examples/ts/file-upload](https://github.com/encoredev/examples/tree/main/ts/file-upload) Handling file uploads and storing file data in a bucket and in a database Create it locally with `encore app create --example=ts/file-upload`. ## Creating a Bucket The core of Object Storage is the **Bucket**, which represents a collection of files. In Encore, buckets must be declared as package level variables, and cannot be created inside functions. Regardless of where you create a bucket, it can be accessed from any service by referencing the variable it's assigned to. When creating a bucket you can configure additional properties, like whether the objects in the bucket should be versioned. For example, to create a bucket for storing profile pictures: ```ts import { Bucket } from "encore.dev/storage/objects"; export const profilePictures = new Bucket("profile-pictures", { versioned: false }); ``` ## Uploading files To upload a file to a bucket, use the `upload` method on the bucket variable. ```ts const data = Buffer.from(...); // image data const attributes = await profilePictures.upload("my-image.jpeg", data, { contentType: "image/jpeg", }); ``` The `upload` method additionally takes an optional `UploadOptions` parameter for configuring additinal options, like setting the content type (see above), or to reject the upload if the object already exists. ## Downloading files To download a file from a bucket, use the `download` method on the bucket variable: ```ts const data = await profilePictures.download("my-image.jpeg"); ``` The `download` method additionally takes a set of options to configure the download, like downloading a specific version if the bucket is versioned. ## Listing objects To list objects in a bucket, use the `list` method on the bucket variable. It returns an async iterator of `ListEntry` objects that you can use to easily iterate over the objects in the bucket using a `for await` loop. For example, to list all profile pictures: ```ts for await (const entry of profilePictures.list({})) { // Do something with entry } ``` The `ListOptions` type can be used to limit the number of objects returned, or to filter them to a specific key prefix. ## Deleting objects To delete an object from a bucket, use the `remove` method on the bucket variable. For example, to delete a profile picture: ```ts await profilePictures.remove("my-image.jpeg"); ``` ## Retrieving object attributes You can retrieve information about an object using the `attrs` method on the bucket variable. It returns the attributes of the object, like its size, content type, and ETag. For example, to get the attributes of a profile picture: ```ts const attrs = await profilePictures.attrs("my-image.jpeg"); ``` For convenience there is also `exists` which returns a boolean indicating whether the object exists. ```ts const exists = await profilePictures.exists("my-image.jpeg"); ``` ## Configuring Public Buckets To configure a bucket to be publicly accessible, set the `public` property to `true` when creating the bucket. This allows objects in the bucket to be accessed via a public URL. For example, to create a public bucket for storing profile pictures: ```ts export const publicProfilePictures = new Bucket("public-profile-pictures", { public: true, versioned: false }); ``` When self-hosting, see how to configure public buckets in the [infrastructure configuration docs](https://encore.dev/docs/self-host/configure-infra). When deploying with the Encore platform it will automatically configure the bucket to be publicly accessible and [configure CDN](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) for optimal content delivery. ### Accessing Public Objects Once a bucket is configured as public, you can access its objects using the `publicUrl` method. This method returns the public URL for the specified object. For example, to get the public URL of a profile picture: ```ts const url = publicProfilePictures.publicUrl("my-image.jpeg"); console.log(`Public URL: ${url}`); ``` ## Error handling The methods throw exceptions if something goes wrong, like if the object doesn't exist or the operation fails. If an object does not exist, it throws an `ObjectNotFound` error. If an upload fails due to a precondition not being met (like if the object already exists and the `notExists: true` option is set), it throws a `PreconditionFailed` error. Other errors are returned as `ObjectsError` errors (which the above errors also extend). ## Bucket references Encore uses static analysis to determine which services are accessing each bucket, and what operations each service is performing. That information is used for features such as rendering architecture diagrams, and is used by the Encore platform to provision infrastructure correctly and configure IAM permissions. This means `Bucket` objects can't be passed around however you like, as it makes static analysis impossible in many cases. To simplify your workflow, given these restrictions, Encore supports defining a "reference" to a bucket that can be passed around any way you want. ### Using bucket references Define a bucket reference by calling `bucket.ref()` from within a service, where `DesiredPermissions` is one of the pre-defined permission types defined in the `encore.dev/storage/objects` module. This means you're effectively pre-declaring the permissions you need, and only the methods that are allowed by those permissions are available on the returned reference object. For example, to get a reference to a bucket that can only download objects: ```typescript import { Uploader } from "encore.dev/storage/objects"; const ref = profilePictures.ref(); // You can now freely pass around `ref`, and you can use // `ref.upload()` just like you would `profilePictures.upload()`. ``` To ensure Encore still is aware of which permissions each service needs, the call to `bucket.ref` must be made from within a service, so that Encore knows which service to associate the permissions with. Encore provides permission interfaces for each operation that can be performed on a bucket: * `Downloader` for downloading objects * `Uploader` for uploading objects * `Lister` for listing objects * `Attrser` for getting object attributes * `Remover` for removing objects * `SignedDownloader` for generating signed download URLs for objects * `SignedUploader` for generating signed upload URLs for objects If you need multiple permissions you can combine them using `&`. For example, `profilePictures.ref` gives you a reference that allows calling both `download` and `upload`. For convenience Encore also provides a `ReadWriter` permission that gives complete read-write access to the bucket, granting all the permissions above. It is equivalent to `Downloader & Uploader & Lister & Attrser & Remover`. ## Signed Upload URLs You can use `signedUploadUrl` to create signed URLs to allow clients to upload content directly into the bucket over the internet. The URL is always restricted to one filename, and has a set expiration date. Anyone in possession of the URL can upload data under this filename without any additional authentication. ```typescript const uploadUrl = await profilePictures.signedUploadUrl("my-user-id", {ttl: 7200}) // Pass url to client ``` The client can now `PUT` to this URL with the content as a binary payload. ```bash curl -X PUT --data-binary @/home/me/dog-wizard.jpeg "https://storage.googleapis.com/profile-pictures/my-user-id/?x-goog-signature=b7a1<...>" ``` ### Why signed upload URLs? Signed URLs are an alternative to accepting the content payload directly in your API. Content upload requests are sometimes inconvenient to handle well: they can be long running and very large. With signed URLs, the content flows directly into the storage bucket, and only object IDs and metadata go through your API service. The trade-off is that the upload flow becomes more complex from a client point of view. ## Signed Download URLs You can use `signedDownloadUrl` to create signed URLs to allow clients to download content directly from the bucket, even if it's private. The URL is always restricted to one filename, and has a set expiration date. Anyone in possession of the URL can download the file without any additional authentication. ```typescript const url = await documents.signedDownloadUrl("letter-1234", {ttl: 7200}) // Pass url to client ``` ### Why signed download URLs? Similar to the upload case, signed download URLs is a way to avoid handing large files or bulk traffic through your API. With signed URLs, the content flows directly from the storage bucket, and only object IDs and metadata go through your API service. Note: unless the content is private, prefer serving urls with `publicUrl()` over signed URLs. Public URLs go over CDN, which is typically significantly more performant and cost effective. --- Source: https://encore.dev/docs/ts/primitives/pubsub Section: Building (Encore.ts) # Pub/Sub > Decoupling services and building asynchronous systems Publishers & Subscribers (Pub/Sub) let you build systems that communicate by broadcasting events asynchronously. This is a great way to decouple services for better reliability and responsiveness. Encore's Backend Framework lets you use Pub/Sub in a cloud-agnostic declarative fashion. At deployment, Encore automatically [provisions the required infrastructure](https://encore.dev/docs/platform/infrastructure/infra). **Related example:** [encoredev/examples/ts/simple-event-driven](https://github.com/encoredev/examples/tree/main/ts/simple-event-driven) Simple example app with an event-driven architecture using Pub/Sub. Create it locally with `encore app create --example=ts/simple-event-driven`. ## Creating a Topic The core of Pub/Sub is the **Topic**, a named channel on which you publish events. Topics must be declared as package level variables, and cannot be created inside functions. Regardless of where you create a topic, it can be published to from any service, and subscribed to from any service. When creating a topic, it must be given an event type, a unique name, and a configuration to define its behaviour. For example, to create a topic with events about user signups: ```ts import { Topic } from "encore.dev/pubsub" export interface SignupEvent { userID: string; } export const signups = new Topic("signups", { deliveryGuarantee: "at-least-once", }); ``` ## Publishing events To publish an **Event**, call `publish` on the topic passing in the event object (which is the type specified in the `new Topic` constructor). For example: ```ts const messageID = await signups.publish({userID: id}); // If we get here the event has been successfully published, // and all registered subscribers will receive the event. // The messageID variable contains the unique id of the message, // which is also provided to the subscribers when processing the event. ``` By defining the `signups` topic variable as an exported variable you can also publish to the topic from other services in the same way. ## Subscribing to Events To **Subscribe** to events, you create a Subscription as a top-level variable, by calling the `new Subscription` constructor. Each subscription needs: - the topic to subscribe to - a name which is unique for the topic - a configuration object with at least a `handler` function to process the events - a configuration object For example, to create a subscription to the `signups` topic from earlier: ```ts import { Subscription } from "encore.dev/pubsub"; const _ = new Subscription(signups, "send-welcome-email", { handler: async (event) => { // Send a welcome email using the event. }, }); ``` Subscriptions can be defined in the same service as the topic is declared, or in any other service of your application. Each subscription to a single topic receives the events independently of any other subscriptions to the same topic. This means that if one subscription is running very slowly, it will grow a backlog of unprocessed events. However, any other subscriptions will still be processing events in real-time as they are published. ### Error Handling If a subscription function returns an error, the event being processed will be retried, based on the retry policy configured on that subscription. After the max number of retries is reached,the event will be placed into a dead-letter queue (DLQ) for that subscriber. This allows the subscription to continue processing events until the bug which caused the event to fail can be fixed. Once fixed, the messages on the dead-letter queue can be manually released to be processed again by the subscriber. ## Customizing message delivery ### At-least-once delivery The above examples configure the topic to ensure that, for each subscription, events will be delivered _at least once_. This means that if the topic believes the event was not processed, it will attempt to deliver the message again. **Therefore, all subscription handlers should be [idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning).** This helps ensure that if the handler is called two or more times, from the outside there's no difference compared to calling it once. This can be achieved using a database to track if you have already performed the action that the event is meant to trigger, or ensuring that the action being performed is also idempotent in nature. ### Exactly-once delivery Topics can also be configured to deliver events _exactly once_ by setting the `deliveryGuarantee` field to `"exactly-once"`. This enables stronger guarantees on the infrastructure level to minimize the likelihood of message re-delivery. However, there are still some rare circumstances when a message might be redelivered. For example, if a networking issue causes the acknowledgement of successful processing the message to be lost before the cloud provider receives it (the [Two Generals' Problem](https://en.wikipedia.org/wiki/Two_Generals%27_Problem)). As such, if correctness is critical under all circumstances, it's still advisable to design your subscription handlers to be idempotent. By enabling exactly-once delivery on a topic the cloud provider enforces certain throughput limitations: - AWS: 300 messages per second for the topic (see [AWS SQS Quotas](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html)). - GCP: At least 3,000 messages per second across all topics in the region (can be higher on the region see [GCP PubSub Quotas](https://cloud.google.com/pubsub/quotas#quotas)). > **Important** > > Exactly-once delivery does not perform message deduplication on the publishing side. > If `publish` is called twice with the same message, the message will be delivered twice. ### Message Attributes By default, each field in the event type is encoded as JSON and sent as part of the Pub/Sub message payload. Pub/Sub topics also support sending data as "attributes", which are key-value pairs that enable other behavior like subscriptions that filter messages or ensuring message ordering. To define that a field should be sent as an attribute, define it with the `Attribute` type. For example, to add an attribute named `source`: ```ts import { Topic, Attribute } from "encore.dev/pubsub"; export interface SignupEvent { userID: string; source: Attribute; } export const signups = new Topic("signups", { deliveryGuarantee: "at-least-once", }); ``` ### Ordered Topics Topics are unordered by default, meaning that messages can be delivered in any order. This allows for better throughput on the topic as messages can be processed in parallel. However, in some cases, messages must be delivered in the order they were published for a given entity. To create an ordered topic, configure the topic's `orderingAttribute` to match the name of a top-level `Attribute` field in the event type. This field ensures that messages delivered to the same subscriber are delivered in the order of publishing for that specific field value. Messages with a different value on the ordering attribute are delivered in an unspecified order. To maintain topic order, messages with the same ordering key aren't delivered until the earliest message is processed or dead-lettered, potentially causing delays due to [head-of-line blocking](https://en.wikipedia.org/wiki/Head-of-line_blocking). Mitigate processing issues by ensuring robust logging and alerts, and appropriate subscription retry policies. > **Note** > > The `orderingAttribute` currently has no effect in local environments. #### Throughput limitations Each cloud provider enforces certain throughput limitations for ordered topics: - **AWS:** 300 messages per second for the topic (see [AWS SQS Quotas](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html)) - **GCP:** 1 MBps for each ordering key (See [GCP Pub/Sub Resource Limits](https://cloud.google.com/pubsub/quotas#resource_limits)) #### Ordered topic example ```ts import { Topic, Attribute } from "encore.dev/pubsub"; export interface CartEvent { shoppingCartID: Attribute; event: string; } export const cartEvents = new Topic("cart-events", { deliveryGuarantee: "at-least-once", orderingAttribute: "shoppingCartID", }) async function example() { // These are delivered in order as they all have the same shopping cart ID await cartEvents.publish({shoppingCartID: 1, event: "item_added"}); await cartEvents.publish({shoppingCartID: 1, event: "checkout_started"}); await cartEvents.publish({shoppingCartID: 1, event: "checkout_completed"}); // This may be delivered at any point as it has a different shopping cart ID. await cartEvents.publish({shoppingCartID: 2, event: "item_added"}); } ``` ## Topic references Encore uses static analysis to determine which services are accessing each Pub/Sub topic, and what operations each service is performing. That information is used for features such as rendering architecture diagrams, and is used by the Encore platform to provision infrastructure correctly and configure IAM permissions. This means `Topic` objects can't be passed around however you like, as it makes static analysis impossible in many cases. To simplify your workflow, given these restrictions, Encore supports defining a "reference" to a topic that can be passed around any way you want. ### Using topic references Define a topic reference by calling `topic.ref()` from within a service, where `DesiredPermissions` is one of the pre-defined permission types defined in the `encore.dev/pubsub` module. This means you're effectively pre-declaring the permissions you need, and only the methods that are allowed by those permissions are available on the returned reference object. For example, to get a reference to a topic that can publish messages: ```typescript import { Publisher } from "encore.dev/pubsub"; const ref = cartEvents.ref(); // You can now freely pass around `ref`, and you can use // `ref.publish()` just like you would `cartEvents.publish()`. ``` To ensure Encore still is aware of which permissions each service needs, the call to `topic.ref` must be made from within a service, so that Encore knows which service to associate the permissions with. Currently, the only permission type is `Publisher`, which allows publishing events to the topic. We plan to add more permission types in the future. --- Source: https://encore.dev/docs/ts/primitives/cron-jobs Section: Building (Encore.ts) # Cron Jobs > Run recurring and scheduled tasks When you need to run periodic and recurring tasks, Encore.ts provides a declarative way of using Cron Jobs. When a Cron Job is defined in your application, Encore automatically calls your specified API according to the defined schedule. This eliminates the need for infrastructure maintenance, as Encore manages scheduling, monitoring, and execution of Cron Jobs. > **Note** > > Cron Jobs do not run when developing locally or in [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments), but you can always call the API manually to test the behavior. **Related example:** [encoredev/examples/ts/uptime](https://github.com/encoredev/examples/tree/main/ts/uptime) Uptime Monitoring app that uses a Cron Job to periodically check the uptime of a website. Create it locally with `encore app create --example=ts/uptime`. ## Defining a Cron Job To define a Cron Job, import `encore.dev/cron` and call `new CronJob`, assigning the result to a top-level variable. ### Example ```ts import { CronJob } from "encore.dev/cron"; import { api } from "encore.dev/api"; // Send a welcome email to everyone who signed up in the last two hours. const _ = new CronJob("welcome-email", { title: "Send welcome emails", every: "2h", endpoint: sendWelcomeEmail, }) // Emails everyone who signed up recently. // It's idempotent: it only sends a welcome email to each person once. export const sendWelcomeEmail = api({}, async () => { // Send welcome emails... }); ``` The `"welcome-email"` argument to `new CronJob` is a unique ID you give to each Cron Job. If you later refactor the code and move the Cron Job definition to another package, Encore uses this ID to keep track that it's the same Cron Job and not a different one. When this code gets deployed Encore will automatically register the Cron Job in the Encore platform and begin calling the `sendWelcomeEmail` API every two hours. The Encore platform dashboard provides a convenient user interface for monitoring and debugging Cron Job executions across all your environments via the `Cron Jobs` menu item: ![Cron Jobs UI](https://encore.dev/assets/docs/cron.png) ## Keep in mind when using Cron Jobs - Cron Jobs do not execute during local development or in [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments). However, you can manually invoke the API to test its behavior. - In the Encore platform, Cron Job executions are limited to **once every hour**, with the exact minute randomized within that hour for users on the Free Tier. To enable more frequent executions or to specify the exact minute within the hour, consider [deploying to your own cloud](https://encore.dev/docs/platform/deploy/own-cloud) or upgrading to the [Pro plan](https://encore.dev/pricing). - Both public and private APIs are supported for Cron Jobs. - Ensure that the API endpoints used in Cron Jobs are idempotent, as they may be called multiple times under certain network conditions. - API endpoints utilized in Cron Jobs must not accept any request parameters. ## Cron schedules Above we used the `every` field, which executes the Cron Job on a periodic basis. It runs around the clock each day, starting at midnight (UTC). In order to ensure a consistent delay between each run, the interval used **must divide 24 hours evenly**. For example, `10m` and `6h` are both allowed (since 24 hours is evenly divisible by both), whereas `7h` is not (since 24 is not evenly divisible by 7). The Encore compiler will catch this and give you a helpful error at compile-time if you try to use an invalid interval. ### Cron expressions For more advanced use cases, such as running a Cron Job on a specific day of the month, or a specific week day, or similar, the `every` field is not expressive enough. For these use cases, Encore provides full support for [Cron expressions](https://en.wikipedia.org/wiki/Cron) by using the `schedule` field instead of the `every` field. For example: ```ts // Run the monthly accounting sync job at 4am (UTC) on the 15th day of each month. const _ = new CronJob("accounting-sync", { title: "Cron Job Example", schedule: "0 4 15 * *", endpoint: accountingSync, }) ``` --- Source: https://encore.dev/docs/ts/primitives/caching Section: Building (Encore.ts) # Caching > Optimize response times and reduce costs by avoiding re-work A cache is a high-speed storage layer, commonly used in distributed systems to improve user experiences by reducing latency, improving system performance, and avoiding expensive computation. For scalable systems you typically want to deploy the cache as a separate infrastructure resource, allowing you to run multiple instances of your application concurrently. Encore's built-in Caching API lets you use high-performance caches (using [Redis](https://redis.io/)) in a cloud-agnostic declarative fashion. At deployment, Encore will automatically [provision the required infrastructure](https://encore.dev/docs/platform/infrastructure/infra). ## Cache clusters To use caching in Encore, you must first define a *cache cluster*. Each cache cluster defined in your application will be provisioned as a separate Redis instance by Encore. This gives you fine-grained control over which service(s) should use the same cache cluster and which should have a separate one. It looks like this: ```typescript import { CacheCluster } from "encore.dev/storage/cache"; const cluster = new CacheCluster("my-cache", { // EvictionPolicy tells Redis how to evict keys when the cache reaches // its memory limit. For typical cache use cases, "allkeys-lru" is a good default. evictionPolicy: "allkeys-lru", }); ``` > **Note** > > When starting out it's recommended to use a single cache cluster > that's shared between your different services. ### Referencing clusters across services To use the same cache cluster from multiple services, use `CacheCluster.named()` to reference an existing cluster by name instead of creating a new one: ```typescript import { CacheCluster, StringKeyspace } from "encore.dev/storage/cache"; // Reference a cluster defined in another service const cluster = CacheCluster.named("my-cache"); const sessions = new StringKeyspace<{ sessionId: string }>(cluster, { keyPattern: "session/:sessionId", }); ``` ### Eviction policies The eviction policy determines how Redis handles keys when the cache reaches its memory limit: - `"allkeys-lru"` - Evicts least recently used keys first (default) - `"noeviction"` - Returns errors when memory limit is reached - `"allkeys-lfu"` - Evicts least frequently used keys first - `"allkeys-random"` - Evicts random keys - `"volatile-lru"` - Evicts least recently used keys with an expiry set - `"volatile-lfu"` - Evicts least frequently used keys with an expiry set - `"volatile-ttl"` - Evicts keys with shortest TTL first - `"volatile-random"` - Evicts random keys with an expiry set ## Keyspaces When using a cache, each cached item is stored at a particular key, which is typically an arbitrary string. If you use a cache cluster to cache different sets of data, it's important that distinct data sets have non-overlapping keys. Each value stored in the cache also has a specific type, and certain cache operations can only be performed on certain types. For example, a common cache operation is to increment an integer value that is stored in the cache. If you try to apply this operation on a value that is not an integer, an error is returned. Encore provides a simple, type-safe solution to these problems through Keyspaces. In order to begin storing data in your cache, you must first define a Keyspace. Each keyspace has a Key type and a Value type. The Key type is much like a map key, in that it tells Encore where in the cache the item is stored. The Key type is combined with the Key Pattern to produce a string that is the Redis cache key. The Value type is the type of the values stored in that keyspace. For many keyspaces this is specified in the name of the constructor. For example, `StringKeyspace` stores `string` values, `IntKeyspace` stores `number` values (as 64-bit integers). ### Example: Rate limiting For example, if you want to rate limit the number of requests per user ID it looks like this: ```typescript import { CacheCluster, IntKeyspace, expireIn } from "encore.dev/storage/cache"; import { api, APIError } from "encore.dev/api"; import { getAuthData } from "~encore/auth"; const cluster = new CacheCluster("rate-limit", { evictionPolicy: "allkeys-lru", }); // RequestsPerUser tracks the number of requests per user. // The cache items expire after 10 seconds without activity. const requestsPerUser = new IntKeyspace<{ userId: string }>(cluster, { keyPattern: "requests/:userId", defaultExpiry: expireIn(10 * 1000), // 10 seconds in milliseconds }); export const myEndpoint = api( { expose: true, method: "GET", path: "/my-endpoint", auth: true }, async (): Promise<{ message: string }> => { const auth = getAuthData(); if (!auth) { throw APIError.unauthenticated("not authenticated"); } const count = await requestsPerUser.increment({ userId: auth.userID }, 1); if (count > 10) { throw APIError.resourceExhausted("rate limit exceeded"); } return { message: "Hello!" }; } ); ``` As you can see, the `requestsPerUser` defines a `keyPattern` which is set to `"requests/:userId"`. Here `:userId` refers to the field in the key type object. When you call `requestsPerUser.increment({ userId: "user123" }, 1)`, Encore generates the Redis key `"requests/user123"`. ### Key patterns with multiple fields You can define key types with multiple fields to create more complex key patterns: ```typescript interface ResourceKey { userId: string; resourcePath: string; } // ResourceRequestsPerUser tracks the number of requests per user and resource. const resourceRequestsPerUser = new IntKeyspace(cluster, { keyPattern: "requests/:userId/:resourcePath", defaultExpiry: expireIn(10 * 1000), }); // Usage: await resourceRequestsPerUser.increment( { userId: "user123", resourcePath: "api/users" }, 1 ); ``` ## Keyspace types Encore comes with several keyspace types, each designed for different use cases: ### StringKeyspace Stores string values. ```typescript import { StringKeyspace } from "encore.dev/storage/cache"; const tokens = new StringKeyspace<{ tokenId: string }>(cluster, { keyPattern: "token/:tokenId", defaultExpiry: expireIn(3600 * 1000), // 1 hour }); // Set a value await tokens.set({ tokenId: "abc123" }, "user-token-value"); // Get a value (returns undefined on cache miss) const token = await tokens.get({ tokenId: "abc123" }); // Delete a value await tokens.delete({ tokenId: "abc123" }); ``` Additional string operations: - `append(key, value)` - Appends to the existing value - `getRange(key, start, end)` - Gets a substring - `setRange(key, offset, value)` - Overwrites part of the string - `len(key)` - Gets the string length ### IntKeyspace Stores 64-bit integer values. Values are floored to integers using `Math.floor`. For fractional values, use `FloatKeyspace` instead. ```typescript import { IntKeyspace } from "encore.dev/storage/cache"; const counters = new IntKeyspace<{ counterId: string }>(cluster, { keyPattern: "counter/:counterId", }); // Set a value await counters.set({ counterId: "visits" }, 0); // Increment and get new value const newCount = await counters.increment({ counterId: "visits" }, 1); // Decrement const decremented = await counters.decrement({ counterId: "visits" }, 1); ``` ### FloatKeyspace Stores 64-bit floating-point values. ```typescript import { FloatKeyspace } from "encore.dev/storage/cache"; const scores = new FloatKeyspace<{ oddsId: string }>(cluster, { keyPattern: "odds/:oddsId", }); // Set a value await scores.set({ oddsId: "game1" }, 1.5); // Increment by a float amount const newOdds = await scores.increment({ oddsId: "game1" }, 0.1); ``` ### StructKeyspace Stores structured data (objects) serialized as JSON. ```typescript import { StructKeyspace } from "encore.dev/storage/cache"; interface UserProfile { name: string; email: string; preferences: { theme: "light" | "dark"; notifications: boolean; }; } const profiles = new StructKeyspace<{ userId: string }, UserProfile>(cluster, { keyPattern: "profile/:userId", defaultExpiry: expireIn(3600 * 1000), }); // Set a structured value await profiles.set( { userId: "user123" }, { name: "Alice", email: "alice@example.com", preferences: { theme: "dark", notifications: true }, } ); // Get the value const profile = await profiles.get({ userId: "user123" }); ``` ### StringListKeyspace Stores ordered lists of string values. ```typescript import { StringListKeyspace } from "encore.dev/storage/cache"; const recentItems = new StringListKeyspace<{ userId: string }>(cluster, { keyPattern: "recent/:userId", }); // Push items to the list await recentItems.pushRight({ userId: "user123" }, "item1", "item2"); // Get items from the list const items = await recentItems.getRange({ userId: "user123" }, 0, -1); // Get all // Pop an item (returns undefined if empty) const lastItem = await recentItems.popRight({ userId: "user123" }); ``` ### NumberListKeyspace Stores ordered lists of numeric values. ```typescript import { NumberListKeyspace } from "encore.dev/storage/cache"; const scoreHistory = new NumberListKeyspace<{ playerId: string }>(cluster, { keyPattern: "scores/:playerId", }); // Push scores await scoreHistory.pushRight({ playerId: "player1" }, 100, 200, 150); // Get all scores const scores = await scoreHistory.items({ playerId: "player1" }); ``` ### StringSetKeyspace Stores unordered sets of unique string values. ```typescript import { StringSetKeyspace } from "encore.dev/storage/cache"; const tags = new StringSetKeyspace<{ articleId: string }>(cluster, { keyPattern: "tags/:articleId", }); // Add members to the set await tags.add({ articleId: "post1" }, "typescript", "encore", "backend"); // Check membership const hasTag = await tags.contains({ articleId: "post1" }, "typescript"); // Get all members const allTags = await tags.items({ articleId: "post1" }); // Remove members await tags.remove({ articleId: "post1" }, "backend"); ``` ### NumberSetKeyspace Stores unordered sets of unique numeric values. ```typescript import { NumberSetKeyspace } from "encore.dev/storage/cache"; const uniqueScores = new NumberSetKeyspace<{ gameId: string }>(cluster, { keyPattern: "unique-scores/:gameId", }); // Add scores await uniqueScores.add({ gameId: "game1" }, 100, 200, 300); // Check if a score exists const has100 = await uniqueScores.contains({ gameId: "game1" }, 100); ``` ## Expiry options Encore provides several ways to set cache entry expiration: ```typescript import { expireIn, expireInSeconds, expireInMinutes, expireInHours, expireDailyAt, neverExpire, keepTTL, } from "encore.dev/storage/cache"; // Expire in milliseconds const expiry1 = expireIn(5000); // 5 seconds // Expire in seconds const expiry2 = expireInSeconds(30); // Expire in minutes const expiry3 = expireInMinutes(5); // Expire in hours const expiry4 = expireInHours(24); // Expire at a specific time each day (UTC) const expiry5 = expireDailyAt(0, 0, 0); // Midnight UTC // Never expire (Redis may still evict based on eviction policy) const expiry6 = neverExpire; // Keep existing TTL when updating (for write operations) const expiry7 = keepTTL; ``` ## Write options When setting values, you can override the default expiry: ```typescript // Set with a specific expiry (overrides default) await keyspace.set(key, value, { expiry: expireInMinutes(30) }); // Keep existing TTL when updating await keyspace.set(key, value, { expiry: keepTTL }); // Only set if key doesn't exist (throws CacheKeyExists otherwise) await keyspace.setIfNotExists(key, value); // Only set if key already exists (throws CacheMiss otherwise) await keyspace.replace(key, value); ``` ## Error handling Cache operations can throw specific error types, all extending the base `CacheError` class: - `CacheMiss` — thrown by `replace()` when the key does not exist. - `CacheKeyExists` — thrown by `setIfNotExists()` when the key already exists. Read operations like `get()` return `undefined` on cache miss instead of throwing. ```typescript import { CacheError, CacheMiss, CacheKeyExists } from "encore.dev/storage/cache"; // get returns undefined on cache miss const value = await keyspace.get(key); if (value === undefined) { // Key doesn't exist in cache } // replace throws CacheMiss if the key doesn't exist try { await keyspace.replace(key, newValue); } catch (err) { if (err instanceof CacheMiss) { console.log("Key doesn't exist, can't replace"); } throw err; } ``` ## Local development For local development, Encore maintains a local, in-memory implementation of Redis. This implementation is designed to store a small amount of keys (currently 100). When the number of keys exceeds this value, keys are randomly purged to get below the limit. This is designed in order to simulate the ephemeral, transient nature of caches while also limiting memory use. The precise behavior for local development may change over time and should not be relied on. --- Source: https://encore.dev/docs/ts/primitives/secrets Section: Building (Encore.ts) # Storing Secrets and API keys > Simply storing secrets securely Wouldn't it be nice to store secret values like API keys, database passwords, and private keys directly in the source code? Of course, we can’t do that – it's horrifyingly insecure! (Unfortunately, it's also [very common](https://www.ndss-symposium.org/ndss-paper/how-bad-can-it-git-characterizing-secret-leakage-in-public-github-repositories/).) Encore's built-in secrets manager makes it simple to store secrets in a secure way and lets you use them in your program like regular variables. **Related example:** [encoredev/examples/ts/simple-event-driven](https://github.com/encoredev/examples/tree/main/ts/simple-event-driven) Simple event driven example that uses secrets to store an API key Create it locally with `encore app create --example=ts/simple-event-driven`. ## Using secrets in your application To use a secret in your application, define a top-level variable directly in your code by calling the `secret` function from `encore.dev/config`. For example: ```ts import { secret } from "encore.dev/config"; // Personal access token for deployments const githubToken = secret("GitHubAPIToken"); // Then, resolve the secret value by calling `githubToken()`. ``` When you've defined a secret in your program, the Encore compiler will check that they are set before running or deploying your application. When running your application locally, if a secret is not set, you will get a warning notifying you that a secret value is missing. When deploying to a cloud environment, all secrets must be defined, otherwise the deploy will fail. Your app's default `staging` environment is a `development` environment, so set a value for it before you push, for example with `encore secret set --type dev,local GitHubAPIToken`. If a deploy fails with `secret key(s) not defined`, set the secret, then redeploy without a new commit: in the Cloud Dashboard, open **Deployments** > **New Deploy**, choose **Branch**, and select the branch the environment deploys from (usually `main`). Once you've provided values for all secrets, call the secret as a function. For example: ```ts async function callGitHub() { const resp = await fetch("https:///api.github.com/user", { credentials: "include", headers: { Authorization: `token ${githubToken()}`, }, }); // ... handle resp } ``` > **Note** > > Secret keys are globally unique for your whole application. If multiple services use the same secret name they both receive the same secret value at runtime. ## Storing secret values ### Using the Encore platform dashboard The simplest way to set up secrets is with the Secrets Manager in the Encore platform Dashboard. Open your app in the [Encore platform dashboard](https://app.encore.dev), go to **Settings** in the main navigation, and then click on **Secrets** in the settings menu. From here you can create secrets, save secret values, and configure different values for different environments. ![Encore's Secrets Manager](https://encore.dev/assets/docs/secrets.png) ### Using the CLI If you prefer, you can also set up secrets from the CLI using:
`encore secret set --type ` `` defines which environment types the secret value applies to. Use a comma-separated list of `production`, `development`, `preview`, and `local`. Shorthands: `prod`, `dev`, `pr`. For example `encore secret set --type prod SSHPrivateKey` sets the secret value for production environments,
and `encore secret set --type dev,preview,local GitHubAPIToken` sets the secret value for development, preview, and local environments. In some cases, it can be useful to define a secret for a specific environment instead of an environment type. You can do so with `encore secret set --env `. Secret values for specific environments take precedence over values for environment types. ### Environment settings Each secret can only have one secret value for each environment type. For example: If you have a secret value that's shared between `development`, `preview` and `local`, and you want to override the value for `local`, you must first edit the existing secret and remove `local` using the Secrets Manager in the [Encore platform dashboard](https://app.encore.dev). You can then add a new secret value for `local`. The end result should look something like the picture below. ![Overriding a secret in Encore's Secrets Manager](https://encore.dev/assets/docs/secretoverride.png) ### Overriding local secrets When setting secrets via the `encore secret set` command, they are automatically synced to all developers working on the same application, courtesy of the Encore Platform. In some cases, however, you want to override a secret only for your local machine. This can be done by creating a file named `.secrets.local.cue` in the root of your Encore application, next to the `encore.app` file. The file contains key-value pairs of secret names to secret values. For example: ```cue -- .secrets.local.cue -- GitHubAPIToken: "my-local-override-token" SSHPrivateKey: "custom-ssh-private-key" ``` ## How it works: Where secrets are stored When you store a secret Encore stores it encrypted using Google Cloud Platform's [Key Management Service](https://cloud.google.com/security-key-management) (KMS). - **Production / Your own cloud:** When you deploy to production using your own cloud account on GCP or AWS, Encore provisions a secrets manager in your account (using either KMS or AWS Secrets Manager) and replicates your secrets to it. The secrets are then injected into the container using secret environment variables. - **Local:** For local secrets Encore automatically replicates them to developers' machines when running `encore run`. - **Development / the Encore platform:** Environments on Encore's development cloud (running on GCP under the hood) work the same as self-hosted GCP environments, using GCP Secrets Manager. --- Source: https://encore.dev/docs/ts/develop/metadata Section: Building (Encore.ts) # Metadata > Use the metadata API to get information about the app and the environment it's running in While Encore tries to provide a cloud-agnostic environment, sometimes it's helpful to know more about the environment your application is running in. For this reason Encore provides an API for accessing metadata about the [application](#application-metadata) and the environment it's running in as part of the `encore.dev` package. ## Application Metadata Calling `appMeta()` from the `encore.dev` package returns an object that contains information about the application, including: - `appId` - the application name. - `apiBaseUrl` - the URL the application API can be publicly accessed on. - `environment` - the [environment](https://encore.dev/docs/platform/deploy/environments) the application is currently running in. - `build` - the revision information of the build from the version control system. - `deploy` - the deployment ID and when this version of the app was deployed. ## Current Request The `currentRequest()` function, also provided by the `encore.dev` module, can be called from anywhere within your application and returns a `Request` object that contains information the current request being processed. The object contains different fields depending on whether the current request is an API call or a Pub/Sub message being processed. ```typescript -- API Call -- /** Describes an API call being processed. */ export interface APICallMeta { /** Specifies that the request is an API call. */ type: "api-call"; /** Describes the API Endpoint being called. */ api: APIDesc; /** The HTTP method used in the API call. */ method: Method; /** * The request URL path used in the API call, * excluding any query string parameters. * For example "/path/to/endpoint". */ path: string; /** * The request URL path used in the API call, * including any query string parameters. * For example "/path/to/endpoint?with=querystring". */ pathAndQuery: string; /** * The parsed path parameters for the API endpoint. * The keys are the names of the path parameters, * from the API definition. * * For example {id: 5}. */ pathParams: Record; /** * The request headers from the HTTP request. * The values are arrays if the header contains multiple values, * either separated by ";" or when the header key appears more than once. */ headers: Record; /** * The parsed request payload, as expected by the application code. * Not provided for raw endpoints or when the API endpoint expects no * request data. */ parsedPayload?: Record; } -- Pub/Sub Message -- /** Describes a Pub/Sub message being processed. */ export interface PubSubMessageMeta { /** Specifies that the request is a Pub/Sub message. */ type: "pubsub-message"; /** The service processing the message. */ service: string; /** The name of the Pub/Sub topic. */ topic: string; /** The name of the Pub/Sub subscription. */ subscription: string; /** * The unique id of the Pub/Sub message. * It is the same id returned by `topic.publish()`. * The message id stays the same across delivery attempts. */ messageId: string; /** * The delivery attempt. The first attempt starts at 1, * and increases by 1 for each retry. */ deliveryAttempt: number; /** * The parsed request payload, as expected by the application code. */ parsedPayload?: Record; } ``` This works automatically as a result of Encore's request tracking. If no request is processed by the caller, which can happen if you call it during service initialization, `currentRequest()` returns `undefined`. ## Example Use Cases ### Using Cloud Specific Services All the [clouds](https://encore.dev/docs/platform/deploy/own-cloud) contain a large number of services, not all of which Encore natively supports. By using information about the [environment](https://encore.dev/docs/platform/deploy/environments), you can define the implementation of these and use different services for each environment's provider. For instance if you are pushing audit logs into a data warehouse, when running on GCP you could use BigQuery, but when running on AWS you could use Redshift, when running locally you could simply write them to a file. ```ts import { appMeta } from "encore.dev"; // Emit an audit event. async function audit(userID: string, event: Record) { const cloud = appMeta().environment.cloud; switch (cloud) { case "aws": return writeIntoRedshift(userID, event); case "gcp": return writeIntoBigQuery(userID, event); case "local": return writeIntoFile(userID, event); default: throw new Error(`unknown cloud: ${cloud}`); } } ``` ### Checking Environment type When implementing a signup system, you may want to skip email verification on user signups when developing the application. Using the `appMeta` API, we can check the environment and decide whether to send an email or simply mark the user as verified upon signup. ```ts import { appMeta } from "encore.dev"; export const signup = api( { expose: true }, async (params: SignupParams): Promise => { // more code... // If this is a testing environment, skip sending the verification email. switch (appMeta().environment.type) { case "test": case "development": await markEmailVerified(userID); break; default: await sendVerificationEmail(userID); break; } // more code... }, ); ``` --- Source: https://encore.dev/docs/ts/develop/testing Section: Building (Encore.ts) # Automated testing > Confidence at speed Encore provides built-in testing tools that make it simple to test your application using a variety of test runners. To run tests with Encore: 1. Configure the `test` command in your `package.json` to use the test runner of your choice. 2. Configure your test runner. 3. Run `encore test` from the CLI. The `encore test` command automatically sets up all necessary infrastructure in test mode before running your tests. **Related example:** [encoredev/examples/ts/uptime](https://github.com/encoredev/examples/tree/main/ts/uptime) Uptime monitoring app with API endpoint unit tests written in Vitest. Create it locally with `encore app create --example=ts/uptime`. ## Recommended Setup: Vitest We recommend [Vitest](https://vitest.dev) as your test runner because it offers: - Fast execution - Native ESM and TypeScript support - Jest API compatibility ### Setting up Vitest 1. Create `vite.config.ts` in your application's root directory: ```ts -- vite.config.ts -- /// import { defineConfig } from "vite"; import path from "path"; export default defineConfig({ resolve: { alias: { "~encore": path.resolve(__dirname, "./encore.gen"), }, }, }); ``` 2. Update your `package.json` to include: ```json -- package.json -- { "scripts": { "test": "vitest" } } ``` You're done! Now you can run your tests with `encore test`. ### Optional: IDE Integration #### VS Code Setup If using Vitest, follow these steps: 1. Install the official Vitest VS Code extension 2. Add to `.vscode/settings.json`: ```json -- .vscode/settings.json -- { "vitest.commandLine": "encore test" } ``` As of Vitest plugin version 0.5 ([issue](https://github.com/vitest-dev/vscode/issues/306)), environment configuration requires an updated approach. The following configuration is required to ensure proper functionality: Update `settings.json` to include: ```json -- .vscode/settings.json -- "vitest.nodeEnv": { // generated with `encore daemon env | grep ENCORE_RUNTIME_LIB | cut -d'=' -f2` "ENCORE_RUNTIME_LIB": "/opt/homebrew/Cellar/encore/1.44.5/libexec/runtimes/js/encore-runtime.node" } ``` When running tests within VSCode, file-level parallel execution must be disabled. Update your `vite.config.ts` as follows: ```typescript -- vite.config.ts -- export default defineConfig({ resolve: { alias: { "~encore": path.resolve(__dirname, "./encore.gen"), }, }, test: { fileParallelism: false, }, }); ``` To improve the performance in CI, you can re-enable the parallel execution by overwriting the config in cli `encore test --fileParallelism=true`. ## Integration Testing Best Practices Encore applications typically focus on integration tests rather than unit tests because: - Encore eliminates most boilerplate code - Your code primarily consists of business logic involving databases and inter-service API calls - Integration tests better verify this type of functionality ### Test Environment Benefits When running tests, Encore automatically: - Sets up separate test databases - Configures databases for optimal test performance by: - Skipping `fsync` - Using in-memory filesystems - Removing durability overhead These optimizations make integration tests nearly as fast as unit tests. --- Source: https://encore.dev/docs/ts/develop/debug Section: Building (Encore.ts) # Debugging Encore makes it easy to debug your application using your favorite IDE. ## Enable debugging mode Run your Encore application with `encore run --debug=break`. This will cause Encore to run your app with the `--inspect-brk` flag, which will pause your application until a debugger is attached. Encore will print the URL to the terminal, which you will use to attach your debugger: ```shell $ encore run --debug=break Your API is running at: http://127.0.0.1:4000 Development Dashboard URL: http://localhost:9400/ai-chat-ts-qhwi Process ID: 38965 Debugger listening on ws://127.0.0.1:9229/473dd95f-e71e-4bf2-9eda-6132dd0d6ae3 ``` (Your process id and url will differ). If you don't want the application to pause on startup, you can use `encore run --debug` instead. This will start the application and wait for a debugger to attach, but it won't pause the application until the debugger is attached. ## Attach your debugger When your Encore application is running, it’s time to attach the debugger. The instructions differ depending on how you would like to debug. If instructions for your editor aren’t listed below, consult your editor for information on how to attach a debugger to a running process. ### Visual Studio Code To debug with VS Code you must first add a debug configuration. Press `Run -> Add Configuration`, choose `Node.js -> Attach`. The generated config should look something like this: ```json { "version": "0.2.0", "configurations": [ { "name": "Attach", "port": 9229, "request": "attach", "skipFiles": [ "/**" ], "type": "node" } ] } ``` Next, open the **Run and Debug** menu in the toolbar on the left, select Attach (the configuration you just created), and then press the green arrow. That’s it! You should be able to set breakpoints and have the Encore application pause when they’re hit like you would expect. ## WebStorm To debug with WebStorm (or any other JetBrains IDE), you must first configure a Node.js Attach configuration. Press `Run -> Edit Configurations`, click the `+` button, and choose `Attach to Node.js/Chrome`. Give it a name and hit `OK`. Now select the configuration you just created and press the green bug. That's it. You should be able to set breakpoints and have the Encore application pause when they’re hit like you would expect. --- Source: https://encore.dev/docs/ts/observability/logging Section: Building (Encore.ts) # Logging > Structured logging helps you understand your application Encore offers built-in support for structured logging, which combines a free-form log message with type-safe key-value pairs. Structured fields can be searched, filtered, grouped, and aggregated without parsing the message text. Logging is integrated with [distributed tracing](https://encore.dev/docs/tracing). A log message emitted while handling a request is recorded in that request's trace, alongside the API calls, database queries, and Pub/Sub operations that Encore captures on its own. Application logs add business context that Encore cannot infer from these operations. ## Usage Import the logger and call one of `error`, `warn`, `info`, `debug`, or `trace`: ```ts import log from "encore.dev/log"; log.info("order created", { orderId: "order_123", customerId: "customer_456", total: 14900, currency: "SEK", }); ``` The first parameter is the message, followed by a single object of key-value pairs. Pass variable data as fields rather than interpolating it into the message: ```ts // Prefer log.info("order created", { orderId, customerId }); // Avoid log.info(`order ${orderId} created for customer ${customerId}`); ``` Interpolated values have to be parsed back out before they can be queried, and they make every message unique, so equivalent events can no longer be grouped or counted. ### Logging errors `error` and `warn` accept an error as their first argument: ```ts try { await chargeCustomer(customerId, amount); } catch (err) { log.error(err, "payment authorization failed", { customerId, amount, currency, }); throw err; } ``` The logger records the original error's type, message, stack trace, and cause. Converting it to `String(err)` discards that information before it reaches the logger. ### Contextual loggers When several messages share the same fields, `log.with()` returns a logger that carries them into every subsequent event: ```ts const logger = log.with({ orderId, customerId }); logger.info("order rejected", { reason: "payment_declined" }); logger.info("order cancellation requested", { requestedBy: "customer" }); ``` Use it for context that belongs to a domain operation and that Encore does not already know, such as an import ID or a tenant. Encore already attaches the service, endpoint, trace ID, and authenticated user ID to every log line; do not repeat these fields in a contextual logger. ## What to log - A business state transition, such as a subscription activating or an order shipping. - A decision the application made, together with the inputs that drove it. - A fallback or retry that allowed an operation to complete. - A failure, at the boundary that owns the recovery decision. - An interaction with an external system that needs domain context to interpret. - A security-relevant or administrative action. ```ts log.info("order rejected", { event: "order.rejected", orderId, reason: "insufficient_inventory", productId, requestedQuantity, availableQuantity, }); ``` ```ts log.warn("serving cached exchange rate", { event: "exchange_rate.fallback_used", currencyPair: "EUR_SEK", cacheAgeSeconds, providerErrorCode, }); ``` ### What Encore already records Encore traces API requests, database queries, service-to-service calls, cache operations, and Pub/Sub activity, and the [local development dashboard](https://encore.dev/docs/dev-dash) renders them as a timeline. Logs that narrate the same execution flow duplicate the trace: ```ts // All redundant with the trace log.info("request started"); log.info("querying database"); log.info("calling payments service"); log.info("request completed"); ``` Logging every successful query or function entry and exit also duplicates information already present in the trace. ## Logs, metrics, and traces - **Metrics** answer how often something happens. Use [counters and gauges](https://encore.dev/docs/ts/observability/metrics) for rates, totals, and levels. - **Traces** answer where the time went and what called what. - **Logs** record what happened, with the identifiers needed to investigate an individual event. Use a counter instead of emitting one `info` log per event when only the number of events is needed, such as for cache hits. ## Log levels **`error`** is for an operation that failed unexpectedly and could not complete: an unavailable dependency, data that violates an invariant, or a background job that has permanently failed. Expected outcomes such as a rejected login, a missing resource, failed validation, or a declined payment should not use `error`. Logging expected outcomes at `error` prevents the level from serving as a reliable alerting signal. **`warn`** is for something unexpected that the application recovered from or continued through in a degraded state: falling back to stale data, ignoring malformed optional input, succeeding on retry, approaching a soft limit. A condition that occurs continuously during normal operation is not a warning. **`info`** is for meaningful, relatively low-volume business events: a user-visible state transition, a scheduled job completing, a configuration change, a batch summary. **`debug`** is for detail that helps an engineer investigate behavior and is not needed to operate the application. **`trace`** is for high-volume internal state, such as each iteration of a matching loop. The minimum level is set by `log_level` in `encore.app` and applies to the whole application. Without this setting, every level is emitted, including `trace`. A `trace` call in frequently executed code is therefore emitted in production unless the level is configured. ## Message and field conventions Use the same message for equivalent events and preserve the variable data in fields. For event families that will be queried, counted, or consumed by automation, add a machine-readable `event` field alongside the human-readable message: ```ts log.info("payment authorized", { event: "payment.authorized", paymentId, orderId, amount, currency, }); ``` Dashboards and alerts built on `event` continue to work if the message changes. Name events using `.`, such as `order.created`, `payment.failed`, and `subscription.cancelled`. Field names should mean the same thing everywhere, and a given field should keep its type across events. `amountMinorUnits: 14900` can be summed and compared; `amount: "149.00 SEK"` has to be parsed first, and the parser breaks on the first entry written in another currency's format. Prefer enum-like values over prose, so `reason: "insufficient_inventory"` instead of a sentence describing the shortage. Encore's own fields use snake_case (`trace_id`, `service`, `endpoint`, `uid`). Application fields using camelCase do not replace them. Use one convention for application fields and one name for each concept. Use `encore.dev/log` rather than `console.log` for application events. Encore does not intercept `console`, so its output skips the structured field handling, never reaches the trace, and arrives as unparsed text when logs are streamed. ## Sensitive data Encore writes log fields without redacting them. Never log passwords or password hashes, API keys, access or refresh tokens, session identifiers, cookies, authorization headers, secret configuration values, or payment card and bank account details. Avoid logging whole request or response objects, which tend to contain several of these at once. Prefer internal identifiers over personal values, so `{ userId }` rather than `{ oldEmail, newEmail }`. When a value is needed for correlation but should not be stored, log a stable non-reversible fingerprint of it instead. Logs are durable production data. They are written to your cloud provider's log store, where access control, retention, exports, and backups govern who can read them. Fields are also recorded in the trace, so a sensitive value may be stored under two separate retention policies. Ordinary application logs are also not an audit log. A formal audit trail usually needs explicit event schemas, defined retention, restricted access, tamper resistance, reliable delivery, and clear actor identity. For compliance-sensitive actions, write a dedicated audit event to durable storage; an application log alongside it is still useful for debugging. ## Log an error once Recording the same failure at every layer produces duplicate entries for one event: ```ts async function loadOrder(id: string) { try { return await db.queryRow`...`; } catch (err) { log.error(err, "failed to load order"); throw err; } } async function createInvoice(id: string) { try { const order = await loadOrder(id); // ... } catch (err) { log.error(err, "failed to create invoice"); // second entry, same failure throw err; } } ``` Log at the layer that owns the decision to retry, fall back, or abort, which is also the layer with the domain context to describe what failed. Lower layers let the error propagate: ```ts async function loadOrder(id: string) { return db.queryRow`...`; } async function createInvoice(orderId: string) { try { const order = await loadOrder(orderId); // ... } catch (err) { log.error(err, "invoice creation failed", { orderId }); throw err; } } ``` A lower layer logs when it suppresses the error, retries, detects a broken internal invariant, or holds information the propagating error will lose. Returning structured error information to the caller is often better than logging on the spot. ## Operation and batch summaries At the end of an operation, record its outcome together with relevant counts and its duration: ```ts log.info("account reconciliation completed", { event: "account.reconciliation_completed", accountId, transactionCount, matchedCount, unmatchedCount, adjustmentsCreated, durationMs, }); ``` For a long-running operation, a start event distinguishes an operation that has not started from one that started but did not finish. The trace records the intermediate steps. A batch should record a summary. Logging each item in a 50,000-item batch produces 50,000 entries: ```ts log.info("user import completed", { event: "user_import.completed", importId, totalCount, succeededCount, skippedCount, failedCount, durationMs, }); ``` Individual failures can still be logged when they need investigation: ```ts log.warn("user import row rejected", { importId, rowNumber, reason: "invalid_email" }); ``` For very large batches, aggregate the failures by reason or log a bounded sample rather than every rejected record. ## Controlling log volume Log volume affects ingestion, storage, indexing, and query costs. High-volume entries in frequently executed code make relevant entries harder to find. Provider responses, database records, and configuration trees may contain large fields, encoded data, or secrets. Log selected metadata instead of the complete value: ```ts // Avoid log.debug("provider response received", { response }); // Prefer log.debug("provider response received", { provider: "example-payments", statusCode: response.status, requestId: response.headers.get("x-request-id"), itemCount: response.data.items.length, }); ``` For collections, log counts and boundary identifiers instead of contents: ```ts log.info("orders selected for reconciliation", { orderCount: orders.length, firstOrderId: orders[0]?.id, lastOrderId: orders.at(-1)?.id, }); ``` Truncate long strings, cap arrays at a few representative entries, and record a byte length instead of a document body: ```ts const MAX_ERROR_BODY_LENGTH = 1_000; log.warn("provider returned an invalid response", { provider: "example-payments", statusCode, responseBodyLength: responseBody.length, responseBodyPreview: responseBody.slice(0, MAX_ERROR_BODY_LENGTH), responseBodyTruncated: responseBody.length > MAX_ERROR_BODY_LENGTH, }); ``` Truncation limits volume but does not remove sensitive data. Review previews for secrets and personal data before logging them. Debug logs may be emitted in every environment and included in configured exports. Apply the same volume limits to `debug` fields as to `info` fields. Logs inside loops, Pub/Sub consumers, and frequently called helpers can produce most of an application's log volume. Review volume by service, level, event name, and deployment version to find these call sites. ## Live-streaming logs Stream logs from any environment directly to your terminal: ```shell $ encore logs --env=prod ``` --- Source: https://encore.dev/docs/ts/observability/metrics Section: Building (Encore.ts) # Metrics > Track custom metrics in your TypeScript application Encore provides built-in support for defining custom metrics in your TypeScript applications. Once defined, metrics are automatically collected and displayed in the Encore platform Dashboard, and can be exported to third-party observability services. See the [Platform metrics documentation](https://encore.dev/docs/platform/observability/metrics) for information about integrations with third-party services like Grafana Cloud and Datadog. ## Defining custom metrics Define custom metrics by importing from [`encore.dev/metrics`](https://encore.dev/docs/ts/runtime/metrics) and creating a new metric using the `Counter`, `CounterGroup`, `Gauge`, or `GaugeGroup` classes. For example, to count the number of orders processed: ```typescript import { Counter } from "encore.dev/metrics"; export const ordersProcessed = new Counter("orders_processed"); function process(order: Order) { // ... ordersProcessed.increment(); } ``` ## Metric types Encore currently supports two metric types: counters and gauges. **Counters** measure the count of something. A counter's value must always increase, never decrease. (Note that the value gets reset to 0 when the application restarts.) Typical use cases include counting the number of requests, the amount of data processed, and so on. **Gauges** measure the current value of something. Unlike counters, a gauge's value can fluctuate up and down. Typical use cases include measuring CPU usage, the number of active instances running of a process, and so on. ### Counter example ```typescript import { Counter } from "encore.dev/metrics"; export const ordersProcessed = new Counter("orders_processed"); function processOrder() { ordersProcessed.increment(); // ... process order } ``` You can also increment by a specific value instead of 1: ```typescript import { Counter } from "encore.dev/metrics"; export const bytesProcessed = new Counter("bytes_processed"); function processData(data: Buffer) { bytesProcessed.increment(data.length); // ... process data } ``` ### Gauge example ```typescript import { Gauge } from "encore.dev/metrics"; export const cpuUsage = new Gauge("cpu_usage"); function updateMetrics() { const usage = getCpuUsage(); // returns a number between 0-100 cpuUsage.set(usage); } ``` Another example tracking active connections: ```typescript import { Gauge } from "encore.dev/metrics"; let activeCount = 0; export const activeConnections = new Gauge("active_connections"); function onConnect() { activeCount++; activeConnections.set(activeCount); } function onDisconnect() { activeCount--; activeConnections.set(activeCount); } ``` ## Defining labels Encore's metrics package provides a type-safe way of attaching labels to metrics. To define labels, create an interface type representing the labels and then use `CounterGroup` or `GaugeGroup`. The labels interface defines the structure of labels, where each property corresponds to a single label. Each property must be of type `string`, `number`, or `boolean`. > **Note** > > When using `number` type for labels, the value will be converted to an integer using `Math.floor()`. ### Counter with labels ```typescript import { CounterGroup } from "encore.dev/metrics"; interface Labels { success: boolean; } export const ordersProcessed = new CounterGroup("orders_processed"); function process(order: Order) { let success = false; try { // ... process order success = true; } catch (err) { success = false; } ordersProcessed.with({ success }).increment(); } ``` ### Gauge with labels ```typescript import { GaugeGroup } from "encore.dev/metrics"; interface ConnectionLabels { region: string; } export const activeConnectionsByRegion = new GaugeGroup("active_connections"); const connectionCounts = new Map(); function onConnect(region: string) { const count = (connectionCounts.get(region) || 0) + 1; connectionCounts.set(region, count); activeConnectionsByRegion.with({ region }).set(count); } function onDisconnect(region: string) { const count = Math.max(0, (connectionCounts.get(region) || 0) - 1); connectionCounts.set(region, count); activeConnectionsByRegion.with({ region }).set(count); } ``` ### Multiple labels You can define multiple labels for a metric: ```typescript import { CounterGroup } from "encore.dev/metrics"; interface JobLabels { jobType: string; priority: number; success: boolean; } export const jobsProcessed = new CounterGroup("jobs_processed"); function processJob(jobType: string, priority: number) { try { // ... process job jobsProcessed.with({ jobType, priority, success: true }).increment(); } catch (err) { jobsProcessed.with({ jobType, priority, success: false }).increment(); } } ``` ## Metric references Encore uses static analysis to determine which services are using each metric, and what operations each service is performing. This means metric objects can't be passed around however you like, as it makes static analysis impossible in many cases. To simplify your workflow, given these restrictions, Encore supports defining a "reference" to a metric that can be passed around any way you want. To create a reference, call the `.ref()` method on any metric: ```typescript import { Counter } from "encore.dev/metrics"; export const ordersProcessed = new Counter("orders_processed"); // Create a reference that can be passed around const metricRef = ordersProcessed.ref(); // Pass the reference to other functions function logMetric(metric: Counter) { metric.increment(); } logMetric(metricRef); ``` This works for all metric types (`Counter`, `CounterGroup`, `Gauge`, and `GaugeGroup`). > **Important** > > Each combination of label values creates a unique time series tracked in memory and stored by the monitoring system. > Using numerous labels can lead to a combinatorial explosion, causing high cloud expenses and degraded performance. > > As a general rule, limit the unique time series to tens or hundreds at most, rather than thousands. --- Source: https://encore.dev/docs/ts/develop/multithreading Section: Building (Encore.ts) # Multithreading > True multithreading for JavaScript applications Encore.ts runs using a high-performance Rust runtime that uses multiple threads to handle incoming requests. The Encore.ts Rust runtime handles virtually everything outside of your core business logic: - Parsing and validating incoming requests - Making API calls to other services - Serializing and writing API responses - Observability integrations like distributed tracing - Infrastructure integrations, like executing database queries, reading and writing from object storage, publishing and consuming messages from Pub/Sub, and more This architecture allows for much higher performance and scalability compared to traditional JavaScript frameworks. By offloading most of this to multithreaded Rust, the single-threaded JavaScript event loop becomes free to focus on executing your core business logic. But for more CPU-intensive workloads, the single-threaded JavaScript event loop can still become a performance bottleneck. For these use cases Encore.ts offers Worker Pooling. With Worker Pooling enabled, Encore.ts starts up multiple NodeJS event loops and load-balances incoming requests across them. This can provide a significant performance boost for CPU-intensive workloads. ![](https://encore.dev/assets/blog/worker-pooling/encore-pooling.png) ## Enabling Worker Pooling To enable Worker Pooling, add `"build": {"worker_pooling": true}` to your `encore.app` file. ## Designing your application to work with Worker Pooling Most application code will work with Worker Pooling without any changes. However, it's important to understand the implications of running in a multi-threaded environment. When utilizing Worker Pooling, Encore.ts will automatically spin up multiple NodeJS isolates (one per CPU) to handle incoming requests. Each NodeJS isolate is a separate JavaScript runtime, with its own event loop and memory space. This means that you cannot rely on global shared state that is shared across all incoming requests, since each request may be handled by a different NodeJS isolate. --- Source: https://encore.dev/docs/ts/primitives/defining-apis Section: Building (Encore.ts) # Defining Type-Safe APIs > Simplifying type-safe API development Encore.ts simplifies creating type-safe, idiomatic TypeScript API endpoints and provides built-in validation for incoming requests. At their core, APIs in Encore.ts are normal `async` functions with request and response data types defined as TypeScript interfaces, which Encore.ts uses to encode API requests to HTTP messages. Encore.ts also parses your source code to understand the request and response schema of each endpoint, automatically handling validation of incoming requests against your schema. ## Defining API endpoints To define an API endpoint, use the `api` function from the `encore.dev/api` module. This function wraps a regular TypeScript async function, designating it as an API endpoint. Encore.ts then generates the necessary boilerplate at compile-time. In the example below, we define the API endpoint `ping` which accepts `POST` requests and is exposed as `hello.ping` (because our service name is `hello`). ```typescript -- hello.ts -- import { api } from "encore.dev/api"; export const ping = api( { method: "POST" }, async (p: PingParams): Promise => { return { message: `Hello ${p.name}!` }; }, ); ``` ### Exposing API endpoints to the outside world When you define an API, by default it is not exposed to the outside world, and it can only be called by other APIs within the same Encore application. To expose an API to the internet, add the `expose: true` field to the options object passed in as the first argument to `api`. - `{ expose: false }` – defines a private API that is never accessible to the outside world. It can only be called from other services in your app and via cron jobs. This is default value if the `expose` field isn't set. - `{ expose: true }` – defines a public API that anybody on the internet can call ### Requiring authentication data To require authentication for an API endpoint, add `auth: true` to the API options. With this option, Encore will first call the authentication handler you've defined to validate the authentication of incoming requests. Setting `auth: true` can also be useful for internal APIs that aren't exposed to the internet. In that case, it means that the internal caller must have valid authentication data associated with its request. Finally, even if an API endpoint does not specify `auth: true`, it will still receive any authentication data that was provided. For more information on defining APIs that require authentication, see the [authentication guide](https://encore.dev/docs/ts/develop/auth). ## API Schemas ### Request and response schemas In the example above we defined an API that uses request and response schemas, where the request data is of type `Params` and the response data of type `Response`. That means we need to define them like so: ```typescript -- hello.ts -- import { api } from "encore.dev/api"; // PingParams is the request data for the Ping endpoint. interface PingParams { name: string; } // PingResponse is the response data for the Ping endpoint. interface PingResponse { message: string; } // hello is an API endpoint that responds with a simple response. export const hello = api( { method: "POST", path: "/hello" }, async (p: PingParams): Promise => { return { message: `Hello ${p.name}!` }; }, ); ``` Request and response schemas are both optional. There are four different ways of defining an API: **Using both request and response data:**
`api({ ... }, async (params: Params): Promise => {});` **Only returning a response:**
`api({ ... }, async (): Promise => {});` **With only request data:**
`api({ ... }, async (params: Params): Promise => {});` **Without any request or response data:**
`api({ ... }, async (): Promise => {});` Alternatively, you can express these using type parameters, since `api` is a generic function: **Using both request and response data:**
`api({ ... }, async (params) => {});` **Only returning a response:**
`api({ ... }, async () => {});` **With only request data:**
`api({ ... }, async (params) => {});` **Without any request or response data:**
`api({ ... }, async () => {});` ### Customizing request and response encoding Encore parses the source code to understand the request and response schema of each endpoint. By default, the data is parsed as a JSON body for incoming requests, and written back as JSON responses. This can be customized on a per-field basis, allowing individual fields to be parsed from query strings and HTTP headers with ease. This is done by using the `Header` and `Query` types defined in the `encore.dev/api` module. ### Headers Headers are defined by setting the field type to `Header<"Name-Of-Header">`. It can be used in both request and response data types. In the example below, the `language` field will be fetched from the `Accept-Language` HTTP header. ```ts import { Header } from "encore.dev/api"; interface Params { language: Header<"Accept-Language">; // parsed from header author: string; // not a header } ``` ### Query parameters For `GET`, `HEAD` and `DELETE` requests, parameters are read from the query string by default, since those HTTP methods do not support request bodies. For other HTTP methods (that do support request bodies), parameters are by default read from the HTTP request body as JSON. In those cases, the `Query` type can be used to specify that a field should be parsed from the query string instead. Query strings are not supported in HTTP responses, and are treated as being part of the HTTP response body in JSON. In the example below, the `limit` field will be read from the `limit` query parameter for all HTTP methods, whereas the `author` field will be parsed from the query string only if the method of the request is `GET`, `HEAD` or `DELETE` (and otherwise from the HTTP request body as JSON). ```ts interface Params { limit: Query; // always a query parameter author: string; // query if GET, HEAD or DELETE, otherwise body parameter } ``` ### Path parameters Path parameters are specified by the `path` field in the API Options in `api` call. To specify a placeholder variable, use `:name` and add a function parameter with the same name to the function signature. Encore parses the incoming request URL and makes sure it matches the type of the parameter. The last segment of the path can be parsed as a wildcard parameter by using `*name` with a matching function parameter. Each path parameter (whether a single segment like `:name` or a wildcard parameter like `*name`) must have a matching field in the request data type. For example: ```ts // Retrieves a blog post by its id. export const getBlogPost = api( { method: "GET", path: "/blog/:id/*path" }, async (params: {id: number; path: string}): Promise { // Use id and path to query database... } ) ``` ### Cookie parameters Cookies are defined by setting the field type to `Cookie<"Name-Of-Cookie">`. It can be used in both request and response data types. In the example below we define an optional field that will be parsed from the cookie header. ```ts import { Cookie } from "encore.dev/api"; interface Params { settings?: Cookie<"settings">; // parsed from cookie header } ``` Read more about cookies [here](https://encore.dev/docs/ts/primitives/cookies) ### Fallback routes Encore supports defining fallback routes that will be called if no other endpoint matches the request, using the syntax `path: "/!fallback"`. This is often useful when migrating an existing backend service over to Encore, as it allows you to gradually migrate endpoints over to Encore while routing the remaining endpoints to the existing HTTP router using a raw endpoint with a fallback route. For example: ```ts // Route all requests to the existing HTTP router if no other endpoint matches. export const fallback = api.raw( { expose: true, method: "*", path: "/!path" }, async (req, resp) { // Call old router } ) ``` ## Custom HTTP status codes By default, Encore automatically sets appropriate HTTP status codes for your API responses. We recommend using these default status codes, but there are situations where you might need to set a custom HTTP status code, such as when porting an existing API that clients depend on for specific status codes. To set a custom HTTP status code, include an `HttpStatus` field in your response interface: ```ts import { api, HttpStatus } from "encore.dev/api"; interface Response { msg: string; status: HttpStatus; } export const endpoint = api( { expose: true, method: "GET", path: "/path" }, async (): Promise => { return { msg: "Hello", status: HttpStatus.Created }; } ); ``` The `HttpStatus` enum includes all standard HTTP status codes like `HttpStatus.OK`, `HttpStatus.Created`, `HttpStatus.BadRequest`, etc. ## Raw endpoints In case you need to operate at a lower abstraction level, Encore supports defining raw endpoints that let you access the underlying HTTP request. This is often useful for things like accepting webhooks. Learn more in the [raw endpoints guide](https://encore.dev/docs/ts/primitives/raw-endpoints). ## Sensitive data When handling sensitive information like API keys, passwords, or personally identifiable information (PII), you may want to prevent these details from appearing in traces. Encore provides the `sensitive` option for API endpoints for this purpose. To mark an endpoint as sensitive, add `sensitive: true` to the API options: ```typescript export const processPayment = api( { expose: true, method: "POST", path: "/payments", sensitive: true }, async (params: PaymentParams): Promise => { return { /* ... */ }; } ); ``` When `sensitive: true` is set, Encore automatically redacts all request and response payloads and excludes HTTP headers from traces. --- Source: https://encore.dev/docs/ts/primitives/api-calls Section: Building (Encore.ts) # API Calls > Making API calls is as simple as making function calls Calling API endpoints between services, i.e. service-to-service calls, looks like regular function calls with Encore.ts. This gives you a simple monolith-like developer experience, even when you use multiple services. The only thing you need to do is import the service you want to call from `~encore/clients` and then call its API endpoints like functions. This works because, when compiling your application, Encore uses [static analysis](https://encore.dev/docs/understanding-encore) to parse all APIs and make them available through the `~encore/clients` module for internal calls. You get all the benefits of function calls, like compile-time checking of all the parameters and auto-completion in your editor, while still allowing the division of code into logical components, services, and systems. ### Example In the example below, we import the service `hello` and call the `ping` endpoint using a function call to `hello.ping`. ```typescript import { hello } from "~encore/clients"; // import 'hello' service export const myOtherAPI = api({}, async (): Promise => { const resp = await hello.ping({ name: "World" }); console.log(resp.message); // "Hello World!" }); ``` ### Service client references Encore uses static analysis to determine which services are calling each other. This means service client objects can't be passed around however you like, as it makes static analysis impossible in many cases. To simplify your workflow, given these restrictions, Encore supports defining a "reference" to a service client that can be passed around any way you want. To create a reference, call the `.ref()` method on any service client: ```typescript import { hello } from "~encore/clients"; // Create a reference that can be passed around const helloRef = hello.ref(); // Pass the reference to other functions function doSomething(client: typeof helloRef) { return client.ping({ name: "World" }); } doSomething(helloRef); ``` **Related example:** [encoredev/examples/ts/simple-event-driven](https://github.com/encoredev/examples/tree/main/ts/simple-event-driven) Simple microservices example application with service-to-service API calls. Create it locally with `encore app create --example=ts/simple-event-driven`. --- Source: https://encore.dev/docs/ts/primitives/raw-endpoints Section: Building (Encore.ts) # Defining Raw Endpoints > Drop down in abstraction to access the raw HTTP request Sometimes you need to operate a lower abstraction than Encore.ts normally provides. For example, you might want to access the underlying HTTP request, often useful for things like accepting webhooks. Encore.ts has you covered using "raw endpoints". To define a raw endpoint, use the `api.raw` function. It works similarly to `api`, but does not accept a request and response schema. Instead, it works like the Node.js `http` module and `Express.js`, where the function receives two parameters: a request object and a response writer. It looks like this: ```ts import { api } from "encore.dev/api"; export const myRawEndpoint = api.raw( { expose: true, path: "/raw", method: "GET" }, async (req, resp) => { resp.writeHead(200, { "Content-Type": "text/plain" }); resp.end("Hello, raw world!"); }, ); ``` It can be called like so: ```shell $ curl http://localhost:4000/raw Hello, raw world! ``` **Related example:** [encoredev/examples/ts/slack-bot](https://github.com/encoredev/examples/tree/main/ts/slack-bot) Slack Bot example application that uses Raw endpoints to accept webhooks. Create it locally with `encore app create --example=ts/slack-bot`. **Related docs:** - [/docs/ts/how-to/file-uploads](https://encore.dev/docs/ts/how-to/file-uploads) --- Source: https://encore.dev/docs/ts/primitives/errors Section: Building (Encore.ts) # API Errors > Returning structured error information from your APIs Encore provides a standardized format of returning errors from API endpoints. It looks like this: ```json // HTTP 404 Not Found { "code": "not_found", "message": "sprocket not found", "details": null } ``` To return this, throw the `APIError` exception that Encore provides in the `encore.dev/api` module, with the appropriate error code: ```typescript import { APIError, ErrCode } from "encore.dev/api"; throw new APIError(ErrCode.NotFound, "sprocket not found"); // or as a shorthand you can also write: throw APIError.notFound("sprocket not found"); ``` ## Error Codes The `ErrCode` type in the `encore.dev/api` module defines error codes for common error scenarios. They are identical to the codes defined by `gRPC` for interoperability. The table below summarizes the error codes. | Code | String | HTTP Status | |-----------------------|-------------------------|---------------------------| | `OK` | `"ok"` | 200 OK | | `Canceled` | `"canceled"` | 499 Client Closed Request | | `Unknown` | `"unknown"` | 500 Internal Server Error | | `InvalidArgument` | `"invalid_argument"` | 400 Bad Request | | `DeadlineExceeded` | `"deadline_exceeded"` | 504 Gateway Timeout | | `NotFound` | `"not_found"` | 404 Not Found | | `AlreadyExists` | `"already_exists"` | 409 Conflict | | `PermissionDenied` | `"permission_denied"` | 403 Forbidden | | `ResourceExhausted` | `"resource_exhausted"` | 429 Too Many Requests | | `FailedPrecondition` | `"failed_precondition"` | 400 Bad Request | | `Aborted` | `"aborted"` | 409 Conflict | | `OutOfRange` | `"out_of_range"` | 400 Bad Request | | `Unimplemented` | `"unimplemented"` | 501 Not Implemented | | `Internal` | `"internal"` | 500 Internal Server Error | | `Unavailable` | `"unavailable"` | 503 Unavailable | | `DataLoss` | `"data_loss"` | 500 Internal Server Error | | `Unauthenticated` | `"unauthenticated"` | 401 Unauthorized | ## Additional details To attach additional structured details to errors, use the `withDetails` method on an `APIError`. The details will be returned with the error to external clients. --- Source: https://encore.dev/docs/ts/primitives/validation Section: Building (Encore.ts) # Validation > Validate incoming requests When receiving incoming requests it's best practice to validate the payload to ensure it meets your expectations and includes all required fields. Encore.ts has request validation built in, designed to work seamlessly with TypeScript. It uses the natural TypeScript types directly to validate incoming requests, so you get the best of both worlds: the clean, concise TypeScript syntax, and runtime schema validation. This means your APIs are type-safe during both runtime and compile-time. Encore.ts makes it easy to define API endpoints that combine data from different sources: some fields from the request body, others from the query parameters, and yet others from the HTTP headers. It looks like this: ```ts import { Header, Query, api } from "encore.dev/api"; interface Request { // Optional query parameter. Parsed from the request URL. limit?: Query; // Custom header that must be set. Parsed from the HTTP headers. myHeader: Header<"X-My-Header">; // Required enum. Parsed from the request body. type: "sprocket" | "widget"; } export const myEndpoint = api( { expose: true, method: "POST", path: "/api" }, async ({ limit, myHeader, type }) => { // ... }, ); ``` In the above example, if a request to the endpoint is missing the `X-My-Header` HTTP header or the `type` field in the request body, Encore will return a `400` Bad Request response. The `limit` query parameter is optional and will be either a number or `undefined` in the endpoint handler function. ## Supported validation types ### String fields ```ts interface Schema { name: string; } ``` ### Number fields The `number` type will accept both int and float values: ```ts interface Schema { age: number; } ``` ### Boolean fields ```ts interface Schema { isHuman: boolean; } ``` ### Array fields You can define fields wit array values of `string`, `number`, `boolean`, `null`: ```ts interface Schema { strings: string[]; numbers: number[]; booleans: boolean[]; nulls: null[]; } ``` You can have an array of objects: ```ts interface Schema { users: { name: string; age: number }[]; } ``` It is also possible to have arrays of multiple types: ```ts interface Schema { values: (string | number)[]; } ``` ### Enum fields String enums can be used in the request schema to validate that a field is one of a set of predefined values. For example: ```ts interface Schema { type: "BLOG_POST" | "COMMENT" } ``` You can also use TypeScript enums: ```ts enum PostType { BlogPost = "BLOG_POST", Comment = "COMMENT" } interface Schema { type: PostType, } ``` The two above examples are equivalent. ### Optional fields You can make a field optional by adding a `?` after the field name: ```ts interface Request { name?: string; } export const myEndpoint = api( { expose: true, method: "POST", path: "/body" }, async (req: Request) => { // req.name is a string or undefined }, ); ``` Request will reach the endpoint handler even if the `name` field is missing. If the field is missing, the value will be `undefined`. ### Nullable fields You can make a field nullable by using the `| null` type: ```ts interface Request { name: string | null; } export const myEndpoint = api( { expose: true, method: "POST", path: "/body" }, async (req: Request) => { // req.name is a string or null }, ); ``` ### Union fields You can define a field that can be one of several types by using a union type: ```ts interface Request { value: string | number | boolean; } export const myEndpoint = api( { expose: true, method: "POST", path: "/body" }, async (req: Request) => { // req.value is a string, number, or boolean }, ); ``` ### Reference schema ```ts interface Schema { str: string; // String int: number; // Number list: number[]; // Array of numbers listOfTypes: (number | string )[]; // Array multiple types nullable: number | null; // Nullable maybe?: string; // Optional multiple: boolean | number | string | { name: string }; // Union enum: "John" | "Foo"; // Enum } ``` ## Value based validation rules Use Encore.ts's composable value based validation for use cases like checking the format of an email address or the length of a string. It uses TypeScript's type system and allows you to define validation rules directly in your type definitions. ### Built-in validation Rules - `Min` / `Max`: Validate minimum/maximum values for numbers - `MinLen` / `MaxLen`: Validate minimum/maximum lengths for strings and arrays - `IsURL` / `IsEmail`: Validate that a string is a URL or email address - `StartsWith` / `EndsWith` / `MatchesRegexp`: Validate that a string matches a specific pattern ### Examples Import validation rules from the `encore.dev/validate` package: ```ts import { Min, Max, MinLen, MaxLen, IsEmail, IsURL } from "encore.dev/validate"; interface Schema { // Number between 3 and 1000 (inclusive) count: number & (Min<3> & Max<1000>); // String between 5 and 20 characters username: string & (MinLen<5> & MaxLen<20>); // Must be either a valid URL or email address contact: string & (IsURL | IsEmail); // Array of up to 10 email addresses recipients: Array & MaxLen<10>; } ``` ### Combining Rules You can combine multiple validation rules using: - `&` (and) to require all rules to pass - `|` (or) to require at least one rule to pass ```ts interface Schema { // Must be both >= 3 and <= 1000 count: number & (Min<3> & Max<1000>); // Must be either a URL or an email contact: string & (IsURL | IsEmail); } ``` ### Performance These validation rules are executed directly in Rust at runtime, before the request reaches your JavaScript code. This provides excellent performance while maintaining type safety at both compile-time and runtime. ## Body By default, the data is parsed as a JSON body for incoming requests: ```ts interface Request { name: string; // Parsed from the JSON body } export const myEndpoint = api( { expose: true, method: "POST", path: "/body" }, async (req) => { // req.name is a string }, ); ``` Here, `name` is a required field in the request body. If the request body is missing the `name` field, Encore will return a `400` Bad Request response. ## Query For HTTP methods that support request bodies, parameters are by default read from the HTTP request body as JSON. In those cases, the `Query` type can be used to specify that a field should be parsed from the query string instead. ```typescript import { api, Query } from "encore.dev/api"; interface Schema { query: Query; // this will be parsed from the '?query=...' parameter in the request url } // A simple API endpoint that echoes the data back. export const echo = api( { method: "POST", path: "/example" }, async (params: Schema) => { // params.query is a string }, ); ``` This API endpoint expects incoming requests to look like this: ```output POST /example?query=hello HTTP/1.1 Content-Type: application/json ``` For `GET`, `HEAD` and `DELETE` requests, parameters are read and validated from the query string by default, since those HTTP methods do not support request bodies. For those methods, the `Query` type is not necessary: ```typescript import { Query } from "encore.dev/api"; interface Schema { limit: Query; // always a query parameter author: string; // query if GET, HEAD or DELETE, otherwise body parameter } ``` ### Nested query fields Using the `Query` type as a nested fields has no effect: ```typescript import { api, Query } from "encore.dev/api"; interface Data { query: Query; // this will be parsed from the '?query=...' parameter in the request url nested: { query2: Query; // Query has no effect inside nested fields }; } export const echo = api( { method: "POST", path: "/nested" }, async (params: Data) => { // ... }, ); ``` Nested query params will be sent as part of the JSON body. The above endpoint expects incoming requests to look like this: ```output POST /nested HTTP/1.1 Content-Type: application/json { "nested": { "query2": "not a query string" } } ``` ## Headers Request headers are defined and validated by setting the field type to `Header<"Name-Of-Header">`. It can be used in both request and response data types. In the example below, the `language` field will be fetched from the `Accept-Language` HTTP header. If the request is missing the `Accept-Language` header, Encore will return a `400` Bad Request response. ```ts import { Header } from "encore.dev/api"; interface Params { language: Header<"Accept-Language">; // parsed from header author: string; // not a header } ``` ### Nested header fields Using the `Header` type as a nested fields has no effect: ```typescript import { api, Header } from "encore.dev/api"; interface Data { header: Header<"X-Header">; // this field will be read from the http header nested: { header2: Header<"X-Other-Header">; // Header has no effect inside nested fields }; } // A simple API endpoint that echoes the data back. export const echo = api( { method: "POST", path: "/nested" }, async (params: Data) => { // ... }, ); ``` Nested headers will be sent as part of the JSON body. The above endpoint expects incoming requests to look like this: ```output POST /nested HTTP/1.1 Content-Type: application/json X-Header: this is a header { "nested": { "header2": "not a header", } } ``` ## Params Dynamic path parameters are also defined in the request schema. The parameter will be parsed from the request URL and made available in the request object: ```ts import { api } from "encore.dev/api"; interface Request { // Required path parameter. Parsed from the request URL. id: string; } export const myEndpoint = api( { expose: true, method: "POST", path: "/user/:id" }, async ({ id }: Request) => { // ... }, ); ``` You can also use the `number` type for path parameters: ```ts interface Request { id: number; } ``` Encore.ts will then try to parse the path parameter as a number. If the path parameter is not a valid number, Encore will return a `400` Bad Request response. ## Combining sources You can combine data from different sources in the same request schema. For example, you can have fields that are parsed from the request body, others from the query parameters, and yet others from the HTTP headers. It looks like this: ```ts import { Header, Query, api } from "encore.dev/api"; interface Request { // Required path parameter. Parsed from the request URL. id: number; // Optional query parameter. Parsed from the request URL. limit?: Query; // Custom header that must be set. Parsed from the HTTP headers. myHeader: Header<"X-My-Header">; // Required enum. Parsed from the request body. type: "sprocket" | "widget"; } export const myEndpoint = api( { expose: true, method: "POST", path: "/user/:id" }, async ({ id, limit, myHeader, type }: Request) => { // ... }, ); ``` ## Errors If the validation is not successful, Encore will return a `400` Bad Request response with a JSON body that contains the error message: ```output HTTP/1.1 400 Bad Request { "code": "invalid_argument", "message": "unable to decode request body", "internal_message": "Error(\"missing field name\", line: 1, column: 18)" } ``` ## Response Encore.ts will not perform runtime validation for response data, but you will get compilation errors if you try to return a value that does not match the expected response type. ### Reusing the request type as the response type You often want to return the same data type that you received in a request. In this case, you can reuse the request type as the response type: ```typescript import { api, Header, Query } from "encore.dev/api"; interface Data { header: Header<"X-Header">; // this field will be read from the http header query: Query; // this will be parsed from the '?query=...' parameter in the request url body: string; // this will be sent as part of the JSON body } // A simple API endpoint that echoes the data back. export const echo = api( { method: "POST", path: "/echo" }, async (params: Data): Promise => { return params; // echo the data back }, ); ``` This API endpoint expects incoming requests to look like this: ```output POST /echo?query=hello HTTP/1.1 Content-Type: application/json X-Header: this is a header { "body": "a body", } ``` For HTTP responses the `Query` type is considered to be part of the JSON response body, since query strings only make sense for incoming requests. Responses returned from this endpoint will be serialized as a HTTP response to looks like this: ```output HTTP/1.1 200 OK Content-Type: application/json X-Header: this is a header { "query": "hello", "body": "a body", } ``` ## Under the hood Encore.ts parses your source code to understand the request and response schema that each API endpoint expects, including things like HTTP headers, query parameters, and so on. The schemas are then processed, optimized, and stored as a Protobuf file. When the Encore.ts Rust runtime starts up, it reads the Protobuf file and pre-computes a request decoder and response encoder, optimized for each API endpoint, using the exact type definition each API endpoint expects. In fact, Encore.ts even handles request validation directly in Rust, ensuring invalid requests never have to even touch the JavaScript layer, mitigating many denial of service attacks. Encore’s understanding of the request schema also improves performance. JavaScript runtimes like Deno and Bun use a similar architecture (in fact, Deno also uses Rust+Tokio+Hyper), but lack Encore’s understanding of the request schema. As a result, they need to hand over the un-processed HTTP requests to the single-threaded JavaScript engine for execution. Encore.ts, on the other hand, handles much more of the request processing inside Rust, and only hands over the decoded request objects. By handling much more of the request life-cycle in multi-threaded Rust, the JavaScript event-loop is freed up to focus on executing application business logic instead of parsing HTTP requests, resulting in a significant performance improvement. --- Source: https://encore.dev/docs/ts/primitives/streaming-apis Section: Building (Encore.ts) # Streaming APIs > How to create APIs that stream data Encore makes it easy to create API endpoints that can stream data to and from your applications. **Related example:** [encoredev/examples/ts/streaming-chat](https://github.com/encoredev/examples/tree/main/ts/streaming-chat) Simple chat app using the Streaming API create a WebSocket stream from a web frontend. Create it locally with `encore app create --example=ts/streaming-chat`. ## Different kinds of stream Encore supports three types of streams, each designed for a specific data flow direction: - [**StreamIn**](#streamin): When you need to stream data into your service. - [**StreamOut**](#streamout): When you need to stream data out from your service. - [**StreamInOut**](#streaminout): When you need to stream data into and out of your service. ## How it works When you connect to a streaming API endpoint, the client and server will do a handshake in the form of a HTTP request. If the server accepts the handshake request, a stream is returned to the client and to the API handler. Under the hood the stream is a WebSocket that can be used to send and receive messages over. Path parameters, query parameters and headers can be passed via the handshake request. The stream returned to the client and to the API handler are typed with the incoming and outgoing message types that you specify in your API. ## Defining streaming APIs Similar to how you can define [RESTful API endpoints](https://encore.dev/docs/ts/primitives/defining-apis) with Encore, you can also easily define type-safe streaming API endpoints. They accept a handshake type, an incoming and an outgoing message type (depending on your choice of stream direction). The type parameters are required for Encore to understand your API. If you don't need any data from the handshake, you can ignore that type, and only specify the incoming and outgoing message types. **Related example:** [encoredev/examples/ts/streaming](https://github.com/encoredev/examples/tree/main/ts/streaming) Example showcases the all different streaming APIs: api.streamIn, api.streamOut, and api.streamInOut Create it locally with `encore app create --example=ts/streaming`. ### StreamIn Use `api.streamIn` when you want to have a stream from client to server, for example if you are uploading something from the client to the server: ```typescript import { api } from "encore.dev/api"; import log from "encore.dev/log"; // Used to pass initial data, optional. interface Handshake { user: string; } // What the clients sends over the stream. interface Message { data: string; done: boolean; } // Returned when the stream is done, optional. interface Response { success: boolean; } export const uploadStream = api.streamIn( { path: "/upload", expose: true }, async (handshake, stream) => { const chunks: string[] = []; try { // The stream object is an AsyncIterator that yields incoming messages. for await (const data of stream) { chunks.push(data.data); // Stop the stream if the client sends a "done" message if (data.done) break; } } catch (err) { log.error(`Upload error by ${handshake.user}:`, err); return { success: false }; } log.info(`Upload complete by ${handshake.user}`); return { success: true }; }, ); ``` For `api.streamIn` you need to specify the incoming message type, the handshake type is optional. You can also specify a optional outgoing type if your API handler responds with some data when it is done with the incoming stream. ```ts api.streamIn( {...}, async (handshake, stream): Promise => {...}) ``` ```ts api.streamIn( {...}, async (handshake, stream) => {...}) ``` ```ts api.streamIn( {...}, async (stream): Promise => {...}) ``` ```ts api.streamIn( {...}, async (stream) => {...}) ``` ### StreamOut Use `api.streamOut` if you want to have a stream of messages from the server to client, for example if you are streaming logs from the server: ```typescript import { api, StreamOut } from "encore.dev/api"; import log from "encore.dev/log"; // Used to pass initial data, optional. interface Handshake { rows: number; } // What the server sends over the stream. interface Message { row: string; } export const logStream = api.streamOut( { path: "/logs", expose: true }, async (handshake, stream) => { try { for await (const row of mockedLogs(handshake.rows, stream)) { // Send the message to the client await stream.send({ row }); } } catch (err) { log.error("Upload error:", err); } }, ); // This function generates an async iterator that yields mocked log rows async function* mockedLogs(rows: number, stream: StreamOut) { for (let i = 0; i < rows; i++) { yield new Promise((resolve) => { setTimeout(() => { resolve(`Log row ${i + 1}`); }, 500); }); } // Close the stream when all logs have been sent await stream.close(); } ``` For `api.streamOut` you need to specify the outgoing message type, the handshake type is optional. ```ts api.streamOut( {...}, async (handshake, stream) => {...}) ``` ```ts api.streamOut( {...}, async (stream) => {...}) ``` ### StreamInOut Use `api.streamInOut` when you want to stream messages in both directions, for example if you are building a chat application: ```typescript import { api } from "encore.dev/api"; interface InMessage { // ... } interface OutMessage { // ... } export const ChatStream = api.streamInOut( { path: "/chat", expose: true }, async (stream) => { for await (const chatMessage of stream) { // Respond to the message by sending something back await stream.send({ /* ... */ }) } } ); ``` For `api.streamInOut` you need to specify both the incoming and outgoing message types, the handshake type is optional. ```ts api.streamInOut( {...}, async (handshake, stream) => {...}) ``` ```ts api.streamInOut( {...}, async (stream) => {...}) ``` ## Handshake When you connect to a streaming API endpoint, the client and server will do a handshake in the form of a HTTP request. For all stream types the handshake type is optional, and only needs to be used whenever you need data from the initial request, such as path parameters, query parameters or headers. Note that if you add a handshake data type you also get two arguments to your handler, one for the handshake data and one for the stream, and if you omit the handshake type you only get the stream. ## Requiring authentication You can use your `authHandler` in the same way as for regular endpoints, just specify `auth: true` in your endpoint options. The auth data will be passed from the client to the server via query parameters or headers in the initial handshake request. After a request has been successfully authenticated, you can access authentication data passed from the `authHandler` by calling `getAuthData()`. See more details in the [auth handler docs](https://encore.dev/docs/ts/develop/auth#authentication-handlers). ## Broadcasting messages To broadcast messages to all connected clients, you can store the streams in a map and iterate over them when a new message is received. If a client disconnects, you can remove the stream from the map. ```ts import { api, StreamInOut } from "encore.dev/api"; // Map to hold all connected streams const connectedStreams: Map< string, StreamInOut > = new Map(); // Object sent from the client to the server when establishing a connection interface HandshakeRequest { id: string; } // Object by both server and client interface ChatMessage { username: string; msg: string; } export const chat = api.streamInOut( { expose: true, auth: false, path: "/chat" }, async (handshake, stream) => { connectedStreams.set(handshake.id, stream); try { // The stream object is an AsyncIterator that yields incoming messages. // The loop will continue as long as the client keeps the connection open. for await (const chatMessage of stream) { for (const [key, val] of connectedStreams) { try { // Send the users message to all connected clients. await val.send(chatMessage); } catch (err) { // If there is an error sending the message, remove the client from the map. connectedStreams.delete(key); } } } } catch (err) { // If there is an error reading from the stream, remove the client from the map. connectedStreams.delete(handshake.id); } // When the client disconnects, remove them from the map. connectedStreams.delete(handshake.id); }, ); ``` ## Connecting with the client Using the [generated client](https://encore.dev/docs/ts/cli/client-generation), you can connect to a streaming API endpoint that have `expose` set to `true`. The client stream acts as an async iterator, allowing you to retrieve messages by simply iterating over it: ```typescript const stream = client.serviceName.endpointName(); for await (const msg of stream) { // Do something with each message } ``` To send messages to the service, use the async `send` method: ```typescript const stream = client.serviceName.endpointName(); await stream.send({ ... }); ``` To handle network errors or do some cleanup after the connection is closed, you can attach event listeners on the underlying socket: ```typescript const stream = client.serviceName.endpointName(); stream.socket.on("error", (event) => { // An error occurred }); stream.socket.on("close", (event) => { // Connection was closed }); ``` ## Service to service streaming Like with [other endpoint types](https://encore.dev/docs/ts/primitives/api-calls) you can easily use streaming between services by importing `~encore/clients`. If you want the stream to only be reachable by other services (and not from the public internet), set the `expose` option to false. Example of using a stream endpoint from a regular api endpoint: ```typescript import { chat } from "~encore/clients"; // import 'chat' service export const myOtherAPI = api({}, async (): Promise => { const stream = await chat.myStreamingEndpoint(); // send a message to the chat service over the stream await stream.send({ msg: "data" }); for await (const msg of stream) { // handle incoming message } }); ``` --- Source: https://encore.dev/docs/ts/primitives/graphql Section: Building (Encore.ts) # GraphQL > Serve a GraphQL API under a Raw endpoint Encore.ts has great support for GraphQL with its type-safe approach to building APIs. Encore's automatic tracing also makes it easy to find and fix performance issues that often arise in GraphQL APIs (like the [N+1 problem](https://hygraph.com/blog/graphql-n-1-problem)). **Related example:** [encoredev/examples/ts/graphql](https://github.com/encoredev/examples/tree/main/ts/graphql) Using Apollo GraphQL together with Encore.ts. Create it locally with `encore app create --example=ts/graphql`. ## Concept To serve a GraphQL API, you can leverage [Raw endpoints](https://encore.dev/docs/ts/primitives/raw-endpoints). Raw endpoints provide direct access to the underlying HTTP request and response objects, enabling integration with a GraphQL library. Below is an outline of the high-level steps required for setup: 1. Take client requests with a Raw endpoint. 2. Pass along the request object (body, headers, query params, etc.) to the GraphQL library. 3. Use the GraphQL library to handle the queries and mutations. 4. Return the GraphQL response from the Raw endpoint. Which GraphQL library you choose is up to you. It should work any GraphQL library that allows you to pass along the request object and get a GraphQL response object back without having to start a new HTTP server. ## Example Here's an example using [Apollo](https://www.apollographql.com/docs/apollo-server/) to create a GraphQL API: ```ts import { HeaderMap } from "@apollo/server"; import { api } from "encore.dev/api"; const { ApolloServer, gql } = require("apollo-server"); import { json } from "node:stream/consumers"; // Type definition schema const typeDefs = gql` ... `; // Resolver functions const resolvers = { // ... }; const server = new ApolloServer({ typeDefs, resolvers }); await server.start(); export const graphqlAPI = api.raw( { expose: true, path: "/graphql", method: "*" }, async (req, res) => { // Make sure the Apollo server is started server.assertStarted("/graphql"); // Extract headers in a format that Apollo understands const headers = new HeaderMap(); for (const [key, value] of Object.entries(req.headers)) { if (value !== undefined) { headers.set(key, Array.isArray(value) ? value.join(", ") : value); } } // Get response from Apollo server const httpGraphQLResponse = await server.executeHTTPGraphQLRequest({ httpGraphQLRequest: { headers, method: req.method!.toUpperCase(), body: await json(req), search: new URLSearchParams(req.url ?? "").toString(), }, context: async () => { return { req, res }; }, }); // Set headers for (const [key, value] of httpGraphQLResponse.headers) { res.setHeader(key, value); } // Set status code res.statusCode = httpGraphQLResponse.status || 200; // Write response if it's complete if (httpGraphQLResponse.body.kind === "complete") { res.end(httpGraphQLResponse.body.string); return; } // Write response in chunks if it's async for await (const chunk of httpGraphQLResponse.body.asyncIterator) { res.write(chunk); } res.end(); }, ); ``` **Related docs:** - [/docs/ts/tutorials/graphql](https://encore.dev/docs/ts/tutorials/graphql) ## Call REST APIs in resolvers It's often a good idea to create REST endpoints for your business logic and let your resolvers forward requests to those endpoints. This has a few benefits: 1. **Getting traces** - Calls to Encore endpoints results in traces being created, even for internal API calls. Having traces makes it easy to find and fix performance issues that often arise in GraphQL APIs (like the [N+1 problem](https://hygraph.com/blog/graphql-n-1-problem)). 2. **Thin resolvers** - By making your REST request/response objects extend the generated GraphQL types, your resolvers will just be thin wrappers around your REST endpoints. 3. **Testing** - You can easily test your resolvers by mocking the API calls. 4. **REST & GraphQL** - You will have both a REST and GraphQL API. Here's an example of how it might look like if you can call a REST API from a resolver: ```graphql -- schema.graphql -- type Query { books: [Book] } type Book { title: String! author: String! } ``` ```ts -- resolver.ts -- // Import the book service from the generated service clients import { book } from "~encore/clients"; import { QueryResolvers } from "../__generated__/resolvers-types"; const queries: QueryResolvers = { books: async () => { // Call book.list to get the list of books const { books } = await book.list(); return books; }, }; export default queries; ``` ```ts -- book.ts -- import { api } from "encore.dev/api"; // Import Book type the generated schema types import { Book } from "../__generated__/resolvers-types"; const db: Book[] = [ { title: "To Kill a Mockingbird", author: "Harper Lee", }, // ... ]; // REST endpoint to get the list of books export const list = api( { expose: true, method: "GET", path: "/books" }, async (): Promise<{ books: Book[] }> => { return { books: db }; }, ); ``` --- Source: https://encore.dev/docs/ts/primitives/static-assets Section: Building (Encore.ts) # Static Assets > How to serve static assets Encore.ts has built-in support for serving static assets (such as images, HTML and CSS files, and JavaScript files). This is particularly useful when you want to serve a static website or a single-page application (SPA) that has been pre-compiled into static files. **Related example:** [encoredev/examples/ts/static-files](https://github.com/encoredev/examples/tree/main/ts/static-files) Example of how to use api.static in to serve static files. Create it locally with `encore app create --example=ts/static-files`. ## API Reference Serving static files in Encore.ts works similarly to regular API endpoints, but using the `api.static` function instead. ```typescript import { api } from "encore.dev/api"; export const assets = api.static( { expose: true, path: "/frontend/*path", dir: "./assets" }, ); ``` This will serve all files in the `./assets` directory under the `/frontend` path prefix. Encore automatically serves `index.html` files at the root of a directory. In the case above, that means that requesting the URL `/frontend` will serve the file `./assets/index.html`, and `/frontend/hello` will serve the file `./assets/hello` or `./assets/hello/index.html` (whichever exists). ### Serving static files at the root By default, Encore requires that API endpoint paths don't conflict with other API endpoints. This can cause problems when you want to serve static files at the root of your domain (such as by setting `path: "/*path"`), since that would conflict with all other paths. To support this use case, Encore allows defining a route as a "fallback route", that gets called only when no other API endpoint matches. Fallback routes use the syntax `!path` instead of `*path`. It looks like this: ```typescript import { api } from "encore.dev/api"; export const assets = api.static( { expose: true, path: "/!path", dir: "./assets" }, ); ``` ### Configuring the 404 response When a file matching the request isn't found, Encore automatically serves a 404 Not Found response. You can customize the response by setting the `notFound` option to specify a file that should be served instead: ```typescript import { api } from "encore.dev/api"; export const assets = api.static( { expose: true, path: "/!path", dir: "./assets", notFound: "./not_found.html" }, ); ``` ## Performance When defining static files, the files are served directly from the Encore.ts Rust Runtime. This means that zero JavaScript code is executed to serve the files, freeing up the Node.js runtime to focus on executing business logic. This dramatically speeds up both the static file serving, as well as improving the latency of your API endpoints. --- Source: https://encore.dev/docs/ts/primitives/cookies Section: Building (Encore.ts) # Working with Cookies > Type-safe cookie handling for web applications Encore.ts provides type-safe cookie handling for your API endpoints. Cookies are commonly used for session management, personalization, and tracking. This guide explains how to use cookies in different contexts within your Encore.ts application. ## Where Cookies Can Be Used Cookies can be utilized in three main contexts within Encore.ts: 1. **Auth handler parameters**: Access cookies during authentication 2. **Response cookies**: Set cookies to be sent back to clients 3. **Request cookies**: Access cookies sent by clients ## Using Cookies in Auth Handlers Cookies can be used in authentication handlers: ```ts import { Cookie, Gateway } from "encore.dev/api"; import { authHandler } from "encore.dev/auth"; // Define auth parameters with cookies interface AuthParams { sessionId: Cookie<"sessionId">; } // Auth handler that uses cookies const auth = authHandler(async ({ sessionId }) => { return validateAndGetUser(sessionId.value); }); // Configure the gateway with the auth handler export const gateway = new Gateway({ authHandler: auth, }); ``` ## Setting Cookies in Responses You can set cookies in your API responses by including them in your response type: ```ts import { api, Cookie } from "encore.dev/api"; // Define a response type with a cookie interface LoginResponse { user: UserData; sessionId: Cookie<"sessionId">; } // Create an API endpoint that sets a cookie export const login = api({ method: "POST", path: "/login", expose: true, }, async (params): Promise => { // Authenticate user const user = await authenticateUser(params.username, params.password); const sessionId = generateSessionId(); // Store session await storeSession(sessionId, user.id); // Return response with cookie return { user, sessionId: { value: sessionId, httpOnly: true, secure: true, sameSite: "Strict", maxAge: 60 * 60 * 24 * 7, // 7 days } }; }); ``` ## Using Cookies in Requests When creating API endpoints, you can access cookies sent by the client by defining them in your parameter type: ```ts import { api } from "encore.dev/api"; // Define a request type with a cookie interface Params { language?: Cookie<"language">; } // Create an API endpoint that uses the cookie export const get = api( { method: "GET", path: "/user/profile", expose: true, }, async ({ language }) => { // Access the cookie value const lang = language?.value ?? "en"; return { msg: `your language: ${lang}` }; }, ); ``` ## Typed Cookie Values Encore.ts allows you to specify the type of cookie values for improved type safety. By default, cookie values are strings, but you can define cookies with different data types. If you omit the type parameter, Encore.ts will automatically use `string` as the type. ### Available Cookie Types You can use the following types for cookie values: - `string`: Text data (default when generic parameter is omitted) - `number`: Numeric values - `boolean`: True/false values - `Date`: Date objects ### How to Specify Cookie Types When defining cookies, you can specify the type using the generic parameter: ```ts interface UserPreferences { // Cookies with different types userId: Cookie; darkMode: Cookie; lastVisit: Cookie; language: Cookie; // Using omitted type parameter (implicitly string) theme: Cookie<"theme">; } export const savePreferences = api({ method: "POST", path: "/user/preferences", expose: true, }, async (params) => { // Type-safe access to cookie values const userId = params.userId.value; // number const isDarkMode = params.darkMode.value; // boolean const lastVisit = params.lastVisit.value; // Date const language = params.language.value; // string const theme = params.theme.value; // string (implicitly typed) // Save preferences... }); ``` ## Cookie Options When setting cookies, you can configure various options: - **expires**: Sets an expiration date - **maxAge**: Sets the cookie lifetime in seconds - **domain**: Specifies which domains can receive the cookie - **path**: Limits the cookie to a specific path - **secure**: Only sends the cookie over HTTPS - **httpOnly**: Makes the cookie inaccessible to JavaScript - **sameSite**: Controls when cookies are sent with cross-site requests - `"Strict"`: Only sent in same-site requests - `"Lax"`: Sent with same-site requests and top-level navigations - `"None"`: Sent with all requests (requires `secure: true`) - **partitioned**: Creates a partitioned cookie using the CHIPS model ## Generated client The generated Encore.ts client does not explicitly handle cookies. Instead, it relies on the browser's built-in cookie handling. When using the client in browser environments, cookies will be automatically included in requests and stored from responses. For cross-site requests, you need to configure the client to include credentials: ```ts // Create a client that includes credentials (cookies) for cross-site requests const client = new Client(Local, { requestInit: { credentials: "include" } }); ``` ## Best Practices 1. Use `httpOnly: true` for cookies containing sensitive data 2. Set `secure: true` for production environments 3. Configure appropriate `sameSite` settings based on your requirements 4. Use the `maxAge` or `expires` options to limit cookie lifetime By following these guidelines, you can effectively leverage cookies in your Encore.ts applications while maintaining security and type safety. --- Source: https://encore.dev/docs/ts/primitives/types Section: Building (Encore.ts) # Types > Types in API schemas When you define APIs in Encore.ts, the TypeScript types you use for request and response data are analyzed to generate your API schema. This schema is used for automatic validation, API documentation, and generating type-safe clients. To ensure your API schema can be properly represented and serialized, Encore.ts reduces complex TypeScript types into basic types that can be represented in JSON. This means your API schemas should use simple, serializable types like strings, numbers, booleans, objects, and arrays. ## Decimal JavaScript's native `number` type uses floating-point arithmetic, which can lead to precision errors when working with decimal values. For example, `0.1 + 0.2` equals `0.30000000000000004` instead of `0.3`. Additionally, JavaScript numbers are limited to values between `Number.MIN_SAFE_INTEGER` and `Number.MAX_SAFE_INTEGER` (approximately ±9 quadrillion). To handle decimal values with arbitrary precision and arbitrarily large numbers, Encore.ts provides the `Decimal` type from `encore.dev/types`. This type is especially useful for financial calculations, prices, scientific computations, or any scenario where exact decimal precision and large number support are required. ### Using Decimal in APIs The `Decimal` type can be used in your API request and response schemas just like any other type: ```typescript import { api } from "encore.dev/api"; import { Decimal } from "encore.dev/types"; interface PaymentRequest { amount: Decimal; currency: string; } interface PaymentResponse { total: Decimal; tax: Decimal; } export const processPayment = api( { expose: true, method: "POST", path: "/payments" }, async (req: PaymentRequest): Promise => { const taxRate = new Decimal("0.15"); // 15% tax const tax = req.amount.mul(taxRate); const total = req.amount.add(tax); return { total, tax }; } ); ``` ### Creating Decimal values You can create a `Decimal` from strings, numbers, or bigints: ```typescript import { Decimal } from "encore.dev/types"; const price1 = new Decimal("19.99"); const price2 = new Decimal(29.99); const price3 = new Decimal(100n); ``` For maximum precision, it's recommended to use string literals when creating `Decimal` values to avoid any floating-point conversion issues. ### Arithmetic operations The `Decimal` type supports basic arithmetic operations: ```typescript const a = new Decimal("10.50"); const b = new Decimal("2.25"); const sum = a.add(b); // 12.75 const difference = a.sub(b); // 8.25 const product = a.mul(b); // 23.625 const quotient = a.div(b); // 4.666666... ``` ## Type compatibility and limitations Encore.ts analyzes your TypeScript types to generate API schemas, but TypeScript's type system is incredibly complex and supports many advanced features. While we continuously add support for new type patterns, not all TypeScript type combinations are currently supported for API schemas. ### Working with ORM types A common scenario where you might encounter type compatibility issues is when using types from ORMs (Object-Relational Mappers) or database libraries directly in your API schemas. These types often use complex features from the TypeScript type system. In such cases, it's often better to create dedicated API types and convert between them and your ORM types. ### Benefits of separate API types Creating dedicated API types instead of reusing ORM types can have several advantages: - **Better API design**: Your API schema doesn't have to match your database schema 1:1. You can expose only the fields that make sense for your API consumers. - **Security**: Avoid accidentally exposing sensitive internal fields like password hashes or soft-delete timestamps. - **Stability**: Changes to your database schema don't automatically affect your API contract. - **Type compatibility**: Avoid issues with complex ORM-specific types that may not be supported in API schemas. If you encounter a type that doesn't work in your API schema, creating a dedicated API type as shown above can be a good approach. In many cases, reusing your database types directly works fine, so use separate API types when it makes sense for your use case. --- Source: https://encore.dev/docs/ts/how-to/file-uploads Section: Building (Encore.ts) # Handling file uploads In this guide you will learn how to handle file uploads from a client in your Encore.ts backend. **Related example:** [encoredev/examples/ts/file-upload](https://github.com/encoredev/examples/tree/main/ts/file-upload) Handling file uploads and storing file data in a database Create it locally with `encore app create --example=ts/file-upload`. ## Storing a single file in a database Breakdown of the example: * We have a [PostgreSQL database](https://encore.dev/docs/ts/primitives/databases) table named `files` with columns `name` and `data` to store the file name and the file data. * We have a [Raw Endpoint](https://encore.dev/docs/ts/primitives/raw-endpoints) to handle file uploads. The endpoint has a `bodyLimit` set to `null` to allow for unlimited file size. * We make use of the [busboy](https://www.npmjs.com/package/busboy) library to help with the file handling. * We convert the file data to a `Buffer` and store the file as a `BYTEA` in the database. ```ts -- upload.ts -- import { api } from "encore.dev/api"; import log from "encore.dev/log"; import busboy from "busboy"; import { SQLDatabase } from "encore.dev/storage/sqldb"; // Define a database named 'files', using the database migrations // in the "./migrations" folder. Encore automatically provisions, // migrates, and connects to the database. export const DB = new SQLDatabase("files", { migrations: "./migrations", }); type FileEntry = { data: any[]; filename: string }; /** * Raw endpoint for storing a single file to the database. * Setting bodyLimit to null allows for unlimited file size. */ export const save = api.raw( { expose: true, method: "POST", path: "/upload", bodyLimit: null }, async (req, res) => { const bb = busboy({ headers: req.headers, limits: { files: 1 }, }); const entry: FileEntry = { filename: "", data: [] }; bb.on("file", (_, file, info) => { entry.filename = info.filename; file .on("data", (data) => { entry.data.push(data); }) .on("close", () => { log.info(`File ${entry.filename} uploaded`); }) .on("error", (err) => { bb.emit("error", err); }); }); bb.on("close", async () => { try { const buf = Buffer.concat(entry.data); await DB.exec` INSERT INTO files (name, data) VALUES (${entry.filename}, ${buf}) ON CONFLICT (name) DO UPDATE SET data = ${buf} `; log.info(`File ${entry.filename} saved`); // Redirect to the root page res.writeHead(303, { Connection: "close", Location: "/" }); res.end(); } catch (err) { bb.emit("error", err); } }); bb.on("error", async (err) => { res.writeHead(500, { Connection: "close" }); res.end(`Error: ${(err as Error).message}`); }); req.pipe(bb); return; }, ); -- migrations/1_create_tables.up.sql -- CREATE TABLE files ( name TEXT PRIMARY KEY, data BYTEA NOT NULL ); ``` ### Frontend ```html

``` ## Handling multiple file uploads When handling multiple file uploads, we can use the same approach as above, but we need to handle multiple files in the busboy event listeners. When storing the files in the database, we loop through the files and save them one by one. ```ts export const saveMultiple = api.raw( { expose: true, method: "POST", path: "/upload-multiple", bodyLimit: null }, async (req, res) => { const bb = busboy({ headers: req.headers }); const entries: FileEntry[] = []; bb.on("file", (_, file, info) => { const entry: FileEntry = { filename: info.filename, data: [] }; file .on("data", (data) => { entry.data.push(data); }) .on("close", () => { entries.push(entry); }) .on("error", (err) => { bb.emit("error", err); }); }); bb.on("close", async () => { try { for (const entry of entries) { const buf = Buffer.concat(entry.data); await DB.exec` INSERT INTO files (name, data) VALUES (${entry.filename}, ${buf}) ON CONFLICT (name) DO UPDATE SET data = ${buf} `; log.info(`File ${entry.filename} saved`); } // Redirect to the root page res.writeHead(303, { Connection: "close", Location: "/" }); res.end(); } catch (err) { bb.emit("error", err); } }); bb.on("error", async (err) => { res.writeHead(500, { Connection: "close" }); res.end(`Error: ${(err as Error).message}`); }); req.pipe(bb); return; }, ); ``` ### Frontend ```html

``` ## Handling large files In order to not run into a **Maximum request length exceeded**-error when uploading large files you might need to adjust the endpoints `bodyLimit`. You can also set the `bodyLimit` to `null` to allow for unlimited file size uploads. If unset it defaults to 2MiB. ## Retrieving files from the database When retrieving files from the database, we can use a GET endpoint to fetch the file data by its name. We can then serve the file back to the client by creating a `Buffer` from the file data and sending it in the response. ```ts import { api } from "encore.dev/api"; import { APICallMeta, currentRequest } from "encore.dev"; export const DB = new SQLDatabase("files", { migrations: "./migrations", }); export const get = api.raw( { expose: true, method: "GET", path: "/files/:name" }, async (req, resp) => { try { const { name } = (currentRequest() as APICallMeta).pathParams; const row = await DB.queryRow` SELECT data FROM files WHERE name = ${name}`; if (!row) { resp.writeHead(404); resp.end("File not found"); return; } const chunk = Buffer.from(row.data); resp.writeHead(200, { Connection: "close" }); resp.end(chunk); } catch (err) { resp.writeHead(500); resp.end((err as Error).message); } }, ); ``` You should now be able to retrieve a file from the database by making a GET request to `http://localhost:4000/files/name-of-file.ext`. --- Source: https://encore.dev/docs/ts/develop/auth Section: Building (Encore.ts) # Authenticating users > Knowing what's what and who's who Almost every application needs to know who's calling it, whether the user represents a person in a consumer-facing app or an organization in a B2B app. Encore supports both use cases in a simple yet powerful way. As described in the docs for [defining APIs](https://encore.dev/docs/ts/primitives/defining-apis), each API endpoint can be marked as requiring authentication, using the option `auth: true` when defining the endpoint. ## Authentication Handlers When an API is defined with `auth: true`, you must define an authentication handler in your application. The authentication handler is responsible for inspecting incoming requests to determine what user is authenticated (if any), and computing any other associated authentication information. The authentication handler is defined similarly to API endpoints, using the `authHandler` function imported from `encore.dev/auth`. Like API endpoints, the authentication handler defines what request information it's interested in, in the form of HTTP headers, query strings, or cookies. A simple authentication handler that inspects the `Authorization` header might look like this: ```ts import { Header, Gateway } from "encore.dev/api"; import { authHandler } from "encore.dev/auth"; // AuthParams specifies the incoming request information // the auth handler is interested in. In this case it only // cares about requests that contain the `Authorization` header. interface AuthParams { authorization: Header<"Authorization">; } // The AuthData specifies the information about the authenticated user // that the auth handler makes available. interface AuthData { userID: string; } // The auth handler itself. export const auth = authHandler( async (params) => { // TODO: Look up information about the user based on the authorization header. return {userID: "my-user-id"}; } ) // Define the API Gateway that will execute the auth handler: export const gateway = new Gateway({ authHandler: auth, }) ``` With this in place, Encore will provision an API Gateway that will process incoming requests to your application, and whenever a request contains an `Authorization` header it will first call the authentication handler to resolve information about the user. **Related example:** [encoredev/examples/ts/clerk](https://github.com/encoredev/examples/tree/main/ts/clerk) Example application showing an auth handler implementation using Clerk. Create it locally with `encore app create --example=ts/clerk`. ### Rejecting authentication If the auth handler returns an `AuthData` object, Encore will consider the request authenticated. To instead _reject_ the request, throw an exception. To signal that the credentials are not valid, throw an `APIError` with code `Unauthenticated`. For example: ```ts import { APIError } from "encore.dev/api"; export const auth = authHandler( async (params) => { throw APIError.unauthenticated("bad credentials"); } ) ``` ## Understanding the Authentication Process Encore's authentication process proceeds in two steps: 1. Determine if the request is authenticated 2. Call the endpoint, if permissible #### Step 1: Determining if the request is authenticated Whenever an incoming request contains any of the authentication parameters (defined by the auth handler), Encore's API Gateway calls the auth handler to resolve the authentication data. This happens regardless of the endpoint the request is for. Importantly, it happens even when calling an endpoint that does not require authentication. There are three possible outcomes from calling the auth handler: 1. If the auth handler succeeds, by returning `AuthData`, the request is considered authenticated. 2. If the auth handler throws an `APIError` with code `Unauthenticated`, the request is considered unauthenticated, exactly as if there was no authentication parameters in the request to begin with. 3. If the auth handler throws any other exception, the API Gateway aborts the request and returns the error to the caller. Finally, if the request does not contain authentication data, the request is considered unauthenticated. #### Step 2: Calling the endpoint, if permissible Once the API Gateway has determined whether the request is authenticated, it checks whether the API Endpoint being called requires authentication data. If it does require authentication, and the request is not authenticated, the API Gateway aborts the request and returns an "unauthenticated" error to the caller. In all other situations, the API Gateway proceeds by calling the target endpoint. If the request was successfully authenticated, the authentication data is passed along to the endpoint, regardless of whether the endpoint requires authentication or not. ## Using auth data If a request has been successfully authenticated, the API Gateway forwards the authentication data to the target endpoint. The endpoint can query the available auth data from the `getAuthData` function, available from the `~encore/auth` module. This module is dynamically generated by Encore to enable type-safe resolution of the auth data. ### Propagating auth data Encore automatically propagates the auth data when you make API calls to other Encore API endpoints using the generated `~encore/clients` package. > **Note** > > If an endpoint calls another endpoint during its processing, and the target endpoint > requires authentication while the original request does not have any authentication data, > the API call will fail with error code `Unauthenticated`. > > This behavior preserves the guarantee that endpoints that > require authentication always have valid authentication data present. ## Overriding auth information You can override the auth data for a specific endpoint when calling it via `~encore/clients` by passing `CallOpts`. Example: ```ts import { svc } from "~encore/clients"; const resp = await svc.endpoint(params, { authData: { userID: "...", userEmail: "..." } }); ``` > **Note** > > Overriding auth data is useful for testing endpoints that require authentication without having to > authenticate the request manually. ## Mocking auth You can mock `getAuthData` with vitest. Example: ```ts import { describe, expect, test, vi } from "vitest"; import * as auth from "~encore/auth"; import { get } from "./hello"; describe("get", () => { test("should combine string with parameter value", async () => { const spy = vi.spyOn(auth, 'getAuthData'); spy.mockImplementation(() => ({ userEmail: "user@email.com" })) const resp = await get({ name: "world" }); expect(resp.message).toBe("Hello world! You are authenticated with user@email.com"); }); }); ``` --- Source: https://encore.dev/docs/ts/develop/middleware Section: Building (Encore.ts) # Middleware > Handling cross-cutting, generic functionality Middleware is a way to write reusable code that runs before, after, or both before and after the handling of API requests, often across several (or all) API endpoints. Middleware is commonly used to implement cross-cutting concerns like [request logging](https://encore.dev/docs/ts/observability/logging), [authentication](https://encore.dev/docs/ts/develop/auth), [tracing](https://encore.dev/docs/tracing), and so on. One of the benefits of Encore.ts is that it handles these common use cases out-of-the-box, so there's no need to write your own middleware. However, when developing applications there's often some use cases where it can be useful to write reusable functionality that applies to multiple API endpoints, and middleware is a good solution for this. Encore provides built-in support for middleware by adding functions to the [Service definitions](https://encore.dev/docs/ts/primitives/services) configuration. Each middleware can be configured with a `target` option to specify what API endpoints it applies to. **Related example:** [encoredev/examples/ts/middleware](https://github.com/encoredev/examples/tree/main/ts/middleware) Example app with two middleware; a rate limiter and one for user authorization. Create it locally with `encore app create --example=ts/middleware`. ## Middleware functions The simplest way to create a middleware is to use the `middleware` helper in `encore.dev/api`, here is an example of a middleware that will run for endpoints that require auth: ```ts import { middleware } from "encore.dev/api"; export default new Service("myService", { middlewares: [ middleware({ target: { auth: true } }, async (req, next) => { // do something before the api handler const resp = await next(req); // do something after the api handler return resp }) ] }); ``` Middleware forms a chain, allowing each middleware to introspect and process the incoming request before handing it off to the next middleware by calling the `next` function that's passed in as an argument. For the last middleware in the chain, calling `next` results in the actual API handler being called. The `req` parameter provides information about the incoming request, it has different fields depending on what kind of handler it is. You can get information about the current request via `req.requestMeta` if the endpoint is a [typed API endpoint](https://encore.dev/docs/ts/primitives/defining-apis) or a [Streaming API endpoint](https://encore.dev/docs/ts/primitives/streaming-apis). For [Streaming API endpoints](https://encore.dev/docs/ts/primitives/streaming-apis) you can also access the stream via `req.stream` method. For [Raw Endpoints](https://encore.dev/docs/ts/primitives/raw-endpoints) you can access the raw request and the raw response via `req.rawRequest` and `req.rawResponse`. The `next` function returns a `HandlerResponse` object which contains the response from the API. Extra response headers can be added using `resp.header.set(key, value)` or `resp.header.add(key, value)`, if the endpoint is a [typed API endpoint](https://encore.dev/docs/ts/primitives/defining-apis). To pass data from middleware to an API handler, you can assign values to `req.data` within the middleware. These values can then be accessed in the handler using `currentRequest()`. Here’s an example: ```ts const mw = middleware(async (req, next) => { // Assign a value to the request req.data.myMiddlewareData = { some: "data" }; return await next(req); }); export const ep = api( { expose: true, method: "GET", path: "/endpoint" }, async () => { const callMeta = currentRequest() as APICallMeta; // Access the value in the API handler const myData = callMeta.middlewareData?.myMiddlewareData; // Use the data as needed }, ); ``` ## Middleware ordering Middleware runs in the order they are defined in the [Service definitions](https://encore.dev/docs/ts/primitives/services) configuration, i.e: ```ts export default new Service("myService", { middlewares: [ first, second, third ], }); ``` ## Targeting APIs The `target` option specifies which endpoints within the service the middleware should run on. If not set, the middleware will run for all endpoints by default. For better performance, use the `target` option instead of filtering within the middleware function. This allows the applicable middleware to be determined per endpoint during startup, reducing runtime overhead. The following options are available for targeting endpoints: - `tags`: A list of tags evaluated with `OR`, meaning the middleware applies to an endpoint if the endpoint has at least one of these tags. - `expose`: A boolean indicating whether the middleware should be applied to endpoints that are exposed or not exposed. - `auth`: A boolean indicating whether the middleware should be applied to endpoints that require authentication or not. - `isRaw`: A boolean indicating whether the middleware should be applied to raw endpoints. - `isStream`: A boolean indicating whether the middleware should be applied to stream endpoints. --- Source: https://encore.dev/docs/ts/develop/api-docs Section: Building (Encore.ts) # API Documentation > Write doc comments that are automatically surfaced across your app Encore parses doc comments from your TypeScript source code and surfaces them in the [Service Catalog](https://encore.dev/docs/service-catalog), [generated API clients](https://encore.dev/docs/ts/cli/client-generation), and OpenAPI specs. This means your documentation stays in your code and is always up to date. Both `//` line comments and `/** */` JSDoc comments are supported. ## Services A service's documentation comes from the comment immediately above the `new Service` call in `encore.service.ts`. ```ts -- encore.service.ts -- import { Service } from "encore.dev/service"; // Payments handles payment processing, billing, // and subscription management. export default new Service("payments"); ``` Service docs appear in the Service Catalog and in generated API clients. ## API endpoints Add a comment above the endpoint function to document it. ```ts import { api } from "encore.dev/api"; // Charge charges the given payment method. // It is idempotent; calling it multiple times // with the same idempotency key has no additional effect. export const charge = api( { expose: true, auth: true, method: "POST", path: "/charge" }, async (params: ChargeParams): Promise => { // ... }, ); ``` In the OpenAPI spec, the first line becomes the `summary` and the remaining lines become the `description`. ## Auth handlers ```ts import { authHandler } from "encore.dev/auth"; // AuthenticateRequest validates the API key provided in the Authorization header // and returns the authenticated user's information. export const auth = authHandler(async (params: AuthParams): Promise => { // ... }); ``` ## Types Document interfaces and their fields with comments above each declaration. ```ts // ChargeParams defines the parameters for the Charge endpoint. interface ChargeParams { // The identifier of the payment method to charge. paymentMethodId: string; // The amount to charge, in the smallest currency unit (e.g. cents). amount: number; // ISO 4217 currency code. currency: string; } ``` Both the type-level and field-level comments are surfaced in generated clients and OpenAPI schemas. ## Infrastructure resources Doc comments are also supported on infrastructure resource declarations. ```ts import { SQLDatabase } from "encore.dev/storage/sqldb"; import { Topic, Subscription } from "encore.dev/pubsub"; import { CronJob } from "encore.dev/cron"; import { CacheCluster, CacheKeyspace } from "encore.dev/storage/cache"; import { Bucket } from "encore.dev/storage/objects"; import { Counter } from "encore.dev/metrics"; import { secret } from "encore.dev/config"; import { Gateway } from "encore.dev/api"; // TransactionDB stores payment transactions and related records. const TransactionDB = new SQLDatabase("transactions", { migrations: "./migrations", }); // PaymentEvents publishes events when payment states change. const PaymentEvents = new Topic("payment-events", { deliveryGuarantee: "at-least-once", }); // ProcessPaymentEvent handles incoming payment events. const _ = new Subscription(PaymentEvents, "process-payment", { handler: handlePaymentEvent, }); // DailySettlement runs the settlement process every day at midnight. const DailySettlement = new CronJob("daily-settlement", { every: "24h", endpoint: settle, }); // PaymentCache caches frequently accessed payment records. const PaymentCache = new CacheCluster("payment-cache", { evictionPolicy: "allkeys-lru", }); // PaymentByID caches payments by their unique identifier. const PaymentByID = new CacheKeyspace(PaymentCache, { keyPattern: "payment/:id", }); // Receipts stores generated receipt PDFs. const Receipts = new Bucket("receipts", {}); // ChargeAmount tracks the total amount charged per currency. const ChargeAmount = new Counter("charge_amount", {}); interface ChargeLabels { // ISO 4217 currency code. currency: string; } // StripeAPIKey is the secret key for the Stripe API. const StripeAPIKey = secret("StripeAPIKey"); // PaymentGateway is the API gateway for the payments service. const PaymentGateway = new Gateway({ authHandler: auth }); ``` These docs appear in the Service Catalog and are available through the [MCP server](https://encore.dev/docs/ai-integration#connect-the-local-mcp-server) for AI-assisted development. ## Where docs appear | Where | What's included | |---|---| | [Service Catalog](https://encore.dev/docs/service-catalog) | Services, endpoints, types, fields | | [Generated clients](https://encore.dev/docs/ts/cli/client-generation) | Services, endpoints, types, fields | | OpenAPI spec | Endpoints (summary + description), types, fields | | [MCP server](https://encore.dev/docs/ai-integration#connect-the-local-mcp-server) | All resource types | --- Source: https://encore.dev/docs/ts/cli/client-generation Section: Building (Encore.ts) # Client Library Generation > Stop writing the same types everywhere Encore makes it simple to write scalable distributed backends by allowing you to make function calls that Encore translates into RPC calls. Encore also generates API clients with interfaces that look like the original Go functions, with the same parameters and response signature as the server. The generated clients are single files that use only the standard functionality of the target language, with full type safety. This allow anyone to look at the generated client and understand exactly how it works. The structure of the generated code varies by language, to ensure it's idiomatic and easy to use, but always includes all publicly accessible endpoints, data structures, and documentation strings. Encore currently supports generating the following clients: - **Go** - Using `net/http` for the underlying HTTP transport. - **TypeScript** - Using the browser `fetch` API for the underlying HTTP client. - **JavaScript** - Using the browser `fetch` API for the underlying HTTP client. - **OpenAPI** - Using the OpenAPI Specification's language-agnostic interface to HTTP APIs. (Experimental) If there's a language you think should be added, please submit a pull request or create a feature request on [GitHub](https://github.com/encoredev/encore/issues/new), or [reach out on Discord](https://encore.dev/discord). > **Important** > > If you ship the generated client to end customers, keep in mind that old clients will continue to be used after you make changes. To prevent issues with the generated clients, avoid making breaking changes in APIs that your clients access.
## Generating a Client To generate a client, use the `encore gen client` command. It generates a type-safe client using the most recent API metadata running in a particular environment for the given Encore application. For example: ```shell # Generate a TypeScript client for calling the hello-a8bc application based on the primary environment encore gen client hello-a8bc --output=./client.ts # Generate a Go client for the hello-a8bc application based on the locally running code encore gen client hello-a8bc --output=./client.go --env=local # Generate an OpenAPI client for the hello-a8bc application based on the primary environment encore gen client hello-a8bc --lang=openapi --output=./openapi.json ``` ### Environment Selection By default, `encore gen client` generates the client based on the version of your application currently running in your local environment. You can change this using the `--env` flag and specifying the environment name. > **Note** > > The generated client can be used with any environment, not just the one it was generated for. However, the APIs, data structures > and marshalling logic will be based on whatever is present and running in that environment at the point in time the client is generated. ### Service filtering By default `encore gen client` outputs code for all services with at least one publicly accessible (or authenticated) API. You can narrow down this set of services by specifying the `--services` (or `-s`) flag. It takes a comma-separated list of service names. For example, to generate a typescript client for the `email` and `users` services, run: ```shell encore gen client --services=email,users -o client.ts ``` ### Output Mode By default the client's code will be output to stdout, allowing you to pipe it into your clipboard, or another tool. However, using `--output` you can specify a file location to write the client to. If output is specified, you do not need to specify the language as Encore will detect the language based on the file extension. ### Example Script You could combine this into a `package.json` file for your Typescript frontend, to allow you to run `npm run gen` in that project to update the client to match the code running in your staging environment. ```json -- package.json -- { "scripts": { // ... "gen": "encore gen client hello-a8bc --output=./client.ts --env=staging" // ... } } ``` ## Using the Client The generated client has all the data structures required as parameters or returned as response values as needed by any of the public or authenticated API's of your Encore application. Each service is exposed as object on the client, with each public or authenticated API exposed as a function on those objects. For instance, if you had a service called `email` with a function `Send`, on the generated client you would call this using; `client.email.Send(...)`. ### Creating an instance When constructing a client, you need to pass a `BaseURL` as the first parameter; this is the URL at which the API can be accessed. The client provides two helpers: - `Local` - This is a constant provided, which will always point at your locally running instance environment. - `Environment("name")` - This is a function which allows you to specify an environment by name However, BaseURL is a string, so if the two helpers do not provide enough flexibility you can pass any valid URL to be used as the BaseURL. ### Authentication If your application has any API's which require [authentication](https://encore.dev/docs/ts/develop/auth), then additional options will generated into the client, which can be used when constructing the client. Just like with API's schemas, the data type required by your application's `auth handler` will be part of the client library, allowing you to set it in two ways: If your credentials won't change during the lifetime of the client, simply passing the authentication data to the client through the `WithAuth` (Go) or `auth` (TypeScript) options. However, if the authentication credentials can change, you can also pass a function which will be called before each request and can return a new instance of the authentication data structure or return the existing instance. ### HTTP Client Override If required, you can override the underlying HTTP implementation with your own implementation. This is useful if you want to perform logging of the requests being made, or route the traffic over a secured tunnel such as a VPN. In Go this can be configured using the `WithHTTPClient` option. You are required to provide an implementation of the `HTTPDoer` interface, which the [http.Client](https://pkg.go.dev/net/http#Client) implements. For TypeScript clients, this can be configured using the `fetcher` option and must conform to the same prototype as the browsers inbuilt [fetch API](https://developer.mozilla.org/en-US/docs/Web/API/fetch). ### Structured Errors Errors created or wrapped using Encore's [`errs package`](https://encore.dev/docs/ts/primitives/errors) will be returned to the client and deserialized as an `APIError`, allowing the client to perform adaptive error handling based on the type of error returned. You can perform a type check on errors caused by calling an API to see if it is an `APIError`, and once cast as an `APIError` you can access the `Code`, `Message` and `Details` fields. For TypeScript Encore generates a `isAPIError` type guard which can be used. The `Code` field is an enum with all the possible values generated in the library, alone with description of when we would expect them to be returned by your API. See the [errors documentation](https://encore.dev/docs/ts/primitives/errors#error-codes) for an online reference of this list. --- Source: https://encore.dev/docs/ts/frontend/request-client Section: Building (Encore.ts) # Request client for the frontend > Get type-safety between your backend and frontend Encore is able to generate frontend request clients (TypeScript or JavaScript). This lets you to keep the request/response types in sync without manual work and assists you in calling the APIs. Generate a client by running: ```bash $ encore gen client --output=./src/client.ts --env= ``` Adding this as a script to your `package.json` is often a good idea to be able to run it whenever a change is made to your Encore API: ```json -- package.json -- { ... "scripts": { ... "generate-client:staging": "encore gen client --output=./src/client.ts --env=staging", "generate-client:local": "encore gen client --output=./src/client.ts --env=local" } } ``` After that you are ready to use the request client in your code. In this example, the frontend is calling the `GetNote` endpoint on the `note` service in order to retrieve a specific meeting note (which has the properties `id`, `cover_url` & `text`): ```ts import Client, { Environment, Local } from "src/client.ts"; // Making request to locally running backend... const client = new Client(Local); // or to a specific deployed environment // const client = new Client(Environment("staging")); // Calling APIs as typesafe functions 🌟 const response = await client.note.GetNote("note-uuid"); console.log(response.id); console.log(response.cover_url); console.log(response.text); ``` See more in the [client generation docs](https://encore.dev/docs/ts/cli/client-generation). ### Asynchronous state management When building something a bit more complex, you will likely need to deal with caching, refetching, and data going stale. [TanStack Query](https://tanstack.com/query/latest) is a popular library that was built to solve exactly these problems and works well with the Encore request client. Here is a simple example of using an Encore request client together with TanStack Query: ```ts import { useQuery, useMutation, useQueryClient, QueryClient, QueryClientProvider, } from '@tanstack/react-query' import Client, { todo } from '../encore-client' // Create a Encore client const encoreClient = new Client(window.location.origin); // Create a react-query client const queryClient = new QueryClient() function App() { return ( // Provide the client to your App ) } function Todos() { // Access the client const queryClient = useQueryClient() // Queries const query = useQuery({ queryKey: ['todos'], queryFn: () => encoreClient.todo.List() }) // Mutations const mutation = useMutation({ mutationFn: (params: todo.AddParams) => encoreClient.todo.Add(params), onSuccess: () => { // Invalidate and refetch queryClient.invalidateQueries({ queryKey: ['todos'] }) }, }) return (
    {query.data?.map((todo) => (
  • {todo.title}
  • ))}
) } render(, document.getElementById('root')) ``` This example assumes that we have a `todo` service with a `List` and `Add` endpoint. When adding the new todo, TanStack Query will automatically invalidate the `todos` query and refetch it. For a real-world example, take a look at the [Uptime Monitoring](https://github.com/encoredev/examples/tree/main/uptime) app which also makes use of TanStack Query's `refetchInterval` option for polling the backend. ### Testing When unit testing a component that interacts with your Encore API you can mock methods on the request client to return a value suitable for the test. This makes your test URL agnostic because you are not intercepting specific requests on the fetch layer. You also get type errors in your tests if the request client gets updated. Here is an example from the [Uptime Monitoring Starter](https://github.com/encoredev/examples/tree/main/uptime) where we are mocking a GET request method and spying on a POST request method: ```ts import { render, waitForElementToBeRemoved } from "@testing-library/react"; import App from "./App"; import { site } from "./client"; import { userEvent } from "@testing-library/user-event"; describe("App", () => { beforeEach(() => { // Return mocked data from the List (GET) endpoint jest .spyOn(site.ServiceClient.prototype, "List") .mockReturnValue(Promise.resolve({ sites: [{ id: 1, url: "test.dev" }] })); // Spy on the Add (POST) endpoint jest.spyOn(site.ServiceClient.prototype, "Add"); }); it("render sites", async () => { render(); await waitForElementToBeRemoved(() => screen.queryByText("Loading...")); // Verify that the List endpoint has been called expect(site.ServiceClient.prototype.List).toBeCalledTimes(1); // Verify that the sites are rendered with our mocked data screen.getAllByText("test.dev"); }); it("add site", async () => { render(); await waitForElementToBeRemoved(() => screen.queryByText("Loading...")); // Interact with the page and add 'another.com' await userEvent.click(screen.getByText("Add website")); await userEvent.type( screen.getByPlaceholderText("google.com"), "another.com", ); await userEvent.click(screen.getByText("Save")); // Verify that the Add endpoint has been called with the correct parameters expect(site.ServiceClient.prototype.Add).toHaveBeenCalledWith({ url: "another.com", }); }); }) ``` > **Note** > > In the example above we need to mock the `List` method on `site.ServiceClient.prototype` because the request client has not > yet been initialized when we're creating the mock. If you have access to the instance of the request client in your test > (which could be the case if you are passing the client around in your components) you can instead do `jest.spyOn(client.site, "List")` > and `expect(client.site.List).toHaveBeenCalled()` which would give you the same result. ## REST vs. GraphQL Encore allows for building backends using both REST and GraphQL, you should pick the approach that suits your use case best. Encore's request client only works for REST APIs so if you choose to build a GraphQL backend you will need to use another request library for your frontend. Take a look at the [GraphQL tutorial](https://encore.dev/docs/ts/tutorials/graphql) for an example of building a GraphQL backend with Encore. **Related example:** [encoredev/examples/ts/graphql](https://github.com/encoredev/examples/tree/main/ts/graphql) Example of how to build an Apollo GraphQL server with Encore.ts, implementing a basic book CRUD service. Create it locally with `encore app create --example=ts/graphql`. --- Source: https://encore.dev/docs/ts/tutorials/rest-api Section: Building (Encore.ts) # Building a REST API > Learn how to build a URL shortener with a REST API and PostgreSQL database In this tutorial you will create a REST API for a URL Shortener service. In a few short minutes, you'll learn how to: * Create REST APIs with Encore * Use PostgreSQL databases * Use the local development dashboard to test your app * Create and run tests This is the end result: _[Interactive editor showing the finished `urlShortenerTS` app — open this page in a browser to see it.]_ > **Note** > > To make it easier to follow along, we've laid out a trail of croissants to guide your way. > Whenever you see a 🥐 it means there's something for you to do. ## 1. Create a service and endpoint Create a new application by running `encore app create` and select `Empty app` as the template. If this is the first time you're using Encore, you'll be asked if you wish to create a free account. This is needed when you want Encore to manage functionality like secrets and handle cloud deployments (which we'll use later on in the tutorial). Now let's create a new `url` service. 🥐 In your application's root folder, create a directory named `url` containing a file named `encore.service.ts`. ```shell $ mkdir url $ touch url/encore.service.ts ``` 🥐 Add the following code to `url/encore.service.ts`: ```ts -- url/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("url"); ``` This is how you define a service with Encore. Encore will now consider files in the `url` directory and all its subdirectories as part of the `url` service. 🥐 Create a new file `url.ts` in the `url` directory: ```shell $ touch url/url.ts ``` 🥐 Add the following code to `url/url.ts`: ```ts -- url/url.ts -- import { api } from "encore.dev/api"; import { randomBytes } from "node:crypto"; interface URL { id: string; // short-form URL id url: string; // complete URL, in long form } interface ShortenParams { url: string; // the URL to shorten } // Shortens a URL. export const shorten = api( { method: "POST", path: "/url", expose: true }, async ({ url }: ShortenParams): Promise => { const id = randomBytes(6).toString("base64url"); return { id, url }; }, ); ``` This sets up the `POST /url` endpoint. 🥐 Let’s see if it works! Start your app by running the following command from your app's root directory: ```shell $ encore run ``` You should see this: ```output Encore development server running! Your API is running at: http://127.0.0.1:4000 Development Dashboard URL: http://localhost:9400/5g288 3:50PM INF registered API endpoint endpoint=shorten path=/url service=url ``` 🥐 Next, call your endpoint from the Local Development Dashboard at [http://localhost:9400](http://localhost:9400) and view a trace of the response. It should look like this: You can also call it from the terminal: ```shell $ curl http://localhost:4000/url -d '{"url": "https://encore.dev"}' ``` You should see this: ```output { "id": "5cJpBVRp", "url": "https://encore.dev" } ``` It works! There’s just one problem... Right now, we’re not actually storing the URL anywhere. That means we can generate shortened IDs but there’s no way to get back to the original URL! We need to store a mapping from the short ID to the complete URL. ## 2. Save URLs in a database Fortunately, Encore makes it really easy to set up a PostgreSQL database to store our data. To do so, we first define a **database schema**, in the form of a migration file. 🥐 Create a new folder named `migrations` inside the `url` folder. Then, inside the `migrations` folder, create an initial database migration file named `001_create_tables.up.sql`. The file name format is important (it must start with `001_` and end in `.up.sql`). ```shell $ mkdir url/migrations $ touch url/migrations/001_create_tables.up.sql ``` 🥐 Add the following contents to the file: ```sql -- url/migrations/001_create_tables.up.sql -- CREATE TABLE url ( id TEXT PRIMARY KEY, original_url TEXT NOT NULL ); ``` 🥐 Next, go back to the `url/url.ts` file and import the `SQLDatabase` class from `encore.dev/storage/sqldb` module by modifying the imports to look like this: ```ts -- url/url.ts -- import { api } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { randomBytes } from "node:crypto"; ``` 🥐 Now, to define the database, create an instance of the `SQLDatabase` class in the `url` service: ```ts HL url/url.ts 4:6 -- url/url.ts -- import { api } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { randomBytes } from "node:crypto"; // 'url' database is used to store the URLs that are being shortened. const db = new SQLDatabase("url", { migrations: "./migrations" }); interface URL { id: string; // short-form URL id url: string; // complete URL, in long form } interface ShortenParams { url: string; // the URL to shorten } // Shortens a URL. export const shorten = api( { method: "POST", path: "/url", expose: true }, async ({ url }: ShortenParams): Promise => { const id = randomBytes(6).toString("base64url"); return { id, url }; }, ); ``` 🥐 Lastly, update the `shorten` function to insert data into the database: ```ts HL url/url.ts 21:26 -- url/url.ts -- import { api } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { randomBytes } from "node:crypto"; // 'url' database is used to store the URLs that are being shortened. const db = new SQLDatabase("url", { migrations: "./migrations" }); interface URL { id: string; // short-form URL id url: string; // complete URL, in long form } interface ShortenParams { url: string; // the URL to shorten } // Shortens a URL. export const shorten = api( { method: "POST", path: "/url", expose: true }, async ({ url }: ShortenParams): Promise => { const id = randomBytes(6).toString("base64url"); await db.exec` INSERT INTO url (id, original_url) VALUES (${id}, ${url}) `; return { id, url }; }, ); ``` > **Note** > > Before running your application, make sure you have [Docker](https://www.docker.com) installed and running. It's required to locally run Encore applications with databases. 🥐 Next, start the application again with `encore run` and Encore automatically sets up your database. (In case your application won't run, check the [databases troubleshooting guide](https://encore.dev/docs/ts/primitives/databases#troubleshooting).) You can verify that the database was created by looking at the **Infra** tab in the local development dashboard at [localhost:9400](http://localhost:9400), which should look like this: ![Infra tab in local development dashboard](https://encore.dev/assets/docs/infra_tab.png) 🥐 Now let's call the API again from the local development dashboard, or from the terminal: ```shell $ curl http://localhost:4000/url -d '{"url": "https://encore.dev"}' ``` 🥐 Finally, let's verify that it was saved in the database. You can do this by checking the trace in the local development dashboard, or you can run `encore db shell url` from the app root directory and inputting `select * from url;`: ```shell $ encore db shell url psql (13.1, server 11.12) Type "help" for help. url=# select * from url; id | original_url ----------+-------------------- zr6RmZc4 | https://encore.dev (1 row) ``` That was easy! ## 3. Add endpoint to retrieve URLs To complete our URL shortener API, let’s add the endpoint to retrieve a URL given its short id. 🥐 Add this endpoint to `url/url.ts`: ```ts HL url/url.ts 0:1 HL url/url.ts 29:39 -- url/url.ts -- import { api, APIError } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { randomBytes } from "node:crypto"; // 'url' database is used to store the URLs that are being shortened. const db = new SQLDatabase("url", { migrations: "./migrations" }); interface URL { id: string; // short-form URL id url: string; // complete URL, in long form } interface ShortenParams { url: string; // the URL to shorten } // Shortens a URL. export const shorten = api( { method: "POST", path: "/url", expose: true }, async ({ url }: ShortenParams): Promise => { const id = randomBytes(6).toString("base64url"); await db.exec` INSERT INTO url (id, original_url) VALUES (${id}, ${url}) `; return { id, url }; }, ); // Get retrieves the original URL for the id. export const get = api( { expose: true, auth: false, method: "GET", path: "/url/:id" }, async ({ id }: { id: string }): Promise => { const row = await db.queryRow` SELECT original_url FROM url WHERE id = ${id} `; if (!row) throw APIError.notFound("url not found"); return { id, url: row.original_url }; } ); ``` Encore uses the `/url/:id` syntax to represent a path with a parameter. The `id` name corresponds to the parameter name in the function signature. In this case it is of type `string`, but you can also use other built-in types like `number` or `boolean` if you want to restrict the values. 🥐 We can make sure it works by reviewing the endpoint in the [Service Catalog](https://encore.dev/docs/service-catalog) in the local development dashboard, where we can call it using the `id` you got in the previous step: You can also call it directly from the terminal: ```shell $ curl http://localhost:4000/url/your-id-from-the-previous-step ``` You should now see this: ```json { "id": "your-id-from-the-previous-step", "url": "https://encore.dev" } ``` It works! That's how you build REST APIs and use PostgreSQL databases in Encore. ## 4. Add a test Before deployment, it is good practice to have tests to assure that the service works properly. Such tests including database access are easy to write. 🥐 Let's start by adding the `vitest` package to your project: ```shell $ npm i --save-dev vitest ``` [Vitest](https://vitest.dev/) is a testing framework that works great with Encore but you can use another TypeScript testing framework if you like. 🥐 Next we need to add a test script to our `package.json`: ```json -- package.json -- "scripts": { "test": "vitest" }, ``` We've prepared a test to check that the whole cycle of shortening the URL, storing and then retrieving the original URL works. 🥐 Save this in a separate file `url/url.test.ts`. ```ts -- url/url.test.ts -- import { describe, expect, test } from "vitest"; import { get, shorten } from "./url"; describe("shorten", () => { test("getting a shortened url should give back the original", async () => { const resp = await shorten({ url: "https://example.com" }); const url = await get({ id: resp.id }); expect(url.url).toBe("https://example.com"); }); }); ``` 🥐 Now run `encore test` to verify that it's working. If you use the local development dashboard ([localhost:9400](http://localhost:9400)), you can even see traces for tests. ## 5. Deploy ### Self-hosting Encore supports building Docker images directly from the CLI, which can then be self-hosted on your own infrastructure of choice. If your app is using infrastructure resources, such as SQL databases, Pub/Sub, or metrics, you will need to configure your Docker image with the necessary configuration. Our URL shortener makes use of a PostgreSQL database, so we'll need to supply a runtime configuration so that our app knows how to connect to the database in the cloud. 🥐 Create a new file `infra-config.json` in the root of your project with the following contents: ```json -- infra-config.json -- { "$schema": "https://encore.dev/schemas/infra.schema.json", "sql_servers": [ { "host": "my-db-host:5432", "databases": { "url": { "username": "my-db-owner", "password": {"$env": "DB_PASSWORD"} } } } ] } ``` The values in this configuration are just examples, you will need to replace them with the correct values for your database. Take a look at our guide for [deploying an Encore app with a PostgreSQL database to Digital Ocean](https://encore.dev/docs/self-host/deploy-digitalocean) for more information. 🥐 Build a Docker image by running `encore build docker --config infra-config.json url-shortener:v1.0`. This will compile your application using the host machine and then produce a Docker image containing the compiled application. 🥐 Upload the Docker image to the cloud provider of your choice and run it. ### The Encore platform The Encore platform provides automated infrastructure and DevOps. Deploy to a free Encore Cloud development environment, or connect your own AWS or GCP account for production. ### Create account Before deploying with the Encore platform, your app needs to be linked to a free Encore account. If you created the app while logged in, it's already linked, and you can move on to the next step. Otherwise, sign up or log in and register the app by following steps 1 to 3 of [Already have a local app?](https://encore.dev/docs/platform/deploy/deploying#already-have-a-local-app), then continue below. ### Commit changes The final step before you deploy is to commit all changes to the project repo. 🥐 Commit the new files to the project's git repo and trigger a deploy to a free Encore Cloud development environment by running: ```shell $ git add -A . $ git commit -m 'Initial commit' $ git push encore ``` Encore will now build and test your app, provision the needed infrastructure, and deploy your application to the cloud. After triggering the deployment, you will see a URL where you can view its progress in the [Encore platform dashboard](https://app.encore.dev). It will look something like: `https://app.encore.dev/$APP_ID/envs/staging/deploys/...` From there you can also see metrics, traces, and connect your own AWS or GCP account to use for production deployment. *Now you have a fully fledged backend running in the cloud, well done!* ### Celebrate with fireworks Now that your app is running in the cloud, let's celebrate with some fireworks: 🥐 In the Cloud Dashboard, open the Command Menu by pressing **Cmd + K** (Mac) or **Ctrl + K** (Windows/Linux). _From here you can easily access all Cloud Dashboard features and for example jump straight to specific services in the Service Catalog or view Traces for specific endpoints._ 🥐 Type `fireworks` in the Command Menu and press enter. Sit back and enjoy the show! ![Fireworks](https://encore.dev/assets/docs/fireworks.jpg) 🥐 A great next step is to [integrate with GitHub](https://encore.dev/docs/platform/integrations/github). Once you've linked with GitHub, Encore will automatically start building and running tests against your Pull Requests. ## What's next Now that you know how to build a backend with a database, you're ready to let your creativity flow and begin building your next great idea! We're excited to hear what you're going to build with Encore, join the pioneering developer community on [Discord](https://encore.dev/discord) and share your story. --- Source: https://encore.dev/docs/ts/tutorials/graphql Section: Building (Encore.ts) # Building a GraphQL API > Learn how to build a GraphQL API using Encore Encore has great support for GraphQL with its type-safe approach to building APIs. Encore's automatic tracing also makes it easy to find and fix performance issues that often arise in GraphQL APIs (like the [N+1 problem](https://hygraph.com/blog/graphql-n-1-problem)). In this tutorial we will build a GraphQL API using [Apollo](https://www.apollographql.com/docs/apollo-server/) and Encore.ts. The final code will look like this: _[Interactive editor showing the finished `graphqlTS` app — open this page in a browser to see it.]_ > **Note** > > To make it easier to follow along, we've laid out a trail of croissants to guide your way. > Whenever you see a 🥐 it means there's something for you to do. ## 1. Create your Encore application 🥐 Create a new application by running `encore app create` and select `Empty app` as the template. If this is the first time you're using Encore, you'll be asked if you wish to create a free account. This is optional, but is needed when you want Encore to manage functionality like secrets and handle cloud deployments (which we'll use later on in the tutorial). ## 2. GraphQL setup First, we need to install the necessary dependencies: 🥐 Update your `package.json` file to look like this: ```json -- package.json -- { "name": "encore-graphql", "private": true, "version": "0.0.1", "license": "MPL-2.0", "type": "module", "scripts": { "generate": "graphql-codegen --config codegen.yml" }, "devDependencies": { "@types/node": "^20.5.7", "typescript": "^5.2.2", "@graphql-codegen/cli": "2.16.5", "@graphql-codegen/typescript": "2.8.8", "@graphql-codegen/typescript-resolvers": "2.7.13" }, "dependencies": { "@apollo/server": "^4.11.0", "encore.dev": "^1.35.3", "graphql": "^16.9.0", "graphql-tag": "^2.12.6" } } ``` 🥐 Run `npm install` to install the dependencies. 🥐 Next, create a `codegen.yml` file in the application root containing: ``` -- codegen.yml -- # This configuration file tells GraphQL Code Generator how to generate types based on our schema. schema: './schema.graphql' generates: # Specify where our generated types should live. ./graphql/__generated__/resolvers-types.ts: plugins: - 'typescript' - 'typescript-resolvers' config: useIndexSignature: true ``` ## 3. Add GraphQL schema Now it's time to define the GraphQL schema. 🥐 Create a `schema.graphql` file in the application root containing: ``` -- schema.graphql -- type Query { books: [Book] } type Book { title: String! author: String! } type AddBookMutationResponse { code: String! success: Boolean! message: String! book: Book } type Mutation { addBook(title: String!, author: String!): AddBookMutationResponse } ``` 🥐 Run the code generation script to generate the resolver types: ```shell $ npm run generate ``` The types will be written to `graphql/__generated__/resolvers-types.ts` and will contain a bunch of types that we can use when implementing the resolvers. ## 4. Create a Book service Let's create a simple book service that we can later query using GraphQL. It's a good idea to to make the GraphQL library query Encore endpoints because that will result in traces being created for each called endpoint. Having tracing makes it easy to find and fix performance issues that often arise in GraphQL APIs. 🥐 In your application's root folder, create a directory named `book` containing a file named `encore.service.ts`. ```shell $ mkdir book $ touch book/encore.service.ts ``` 🥐 Add the following code to `book/encore.service.ts`: ```ts -- book/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("book"); ``` This is how you define a service with Encore. Encore will now consider files in the `book` directory and all its subdirectories as part of the `book` service. 🥐 Next, create a `book/book.ts` file containing: ```ts -- book/book.ts -- import { api, APIError } from "encore.dev/api"; import { Book } from "../graphql/__generated__/resolvers-types"; const db: Book[] = [ { title: "To Kill a Mockingbird", author: "Harper Lee", }, { title: "1984", author: "George Orwell", }, { title: "The Great Gatsby", author: "F. Scott Fitzgerald", }, { title: "Moby-Dick", author: "Herman Melville", }, { title: "Pride and Prejudice", author: "Jane Austen", }, ]; export const list = api( { expose: true, method: "GET", path: "/books" }, async (): Promise<{ books: Book[] }> => { return { books: db }; }, ); // Omit the "__typename" field from the request type AddRequest = Omit, "__typename">; export const add = api( { expose: true, method: "POST", path: "/book" }, async (book: AddRequest): Promise<{ book: Book }> => { if (db.some((b) => b.title === book.title)) { throw APIError.alreadyExists( `Book "${book.title}" is already in database`, ); } db.push(book); return { book }; }, ); ``` The `book` service contains two endpoint, one for listing all books and another to add a new book to the database. Our "database" is hardcoded just to limit the scope of this example. Take a look at the [Using SQL databases](https://encore.dev/docs/ts/primitives/databases) docs to learn how to set up and use a database. We get the `Book` type from the generated resolver types. This will make it easier later when we create the resolver functions. ## 5. Create the GraphQL service Now it's time to create our Encore service that will provide the GraphQL API. 🥐 In the `graphql` directory, add a `encore.service.ts` file with the following content: ```ts -- graphql/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("graphql"); ``` Now, we need to create resolvers that call the `book` service. Since the GraphQL API uses the same types as the Encore API exposes (we import types form `resolvers-types.ts` in `book.ts`), our resolver can just be thin wrapper around out API endpoints. 🥐 Create the directory `resolvers` in the `graphql` directory. In the resolvers directory we want to place three files: `index.ts`, `queries.ts` and `mutations.ts`: ```ts -- resolvers/index.ts -- import { Resolvers } from "../__generated__/resolvers-types"; import Query from "./queries.js"; import Mutation from "./mutations.js"; const resolvers: Resolvers = { Query, Mutation }; export default resolvers; -- resolvers/queries.ts -- import { book } from "~encore/clients"; import { QueryResolvers } from "../__generated__/resolvers-types"; // Use the generated `QueryResolvers` type to type check our queries! const queries: QueryResolvers = { books: async () => { const { books } = await book.list(); return books; }, }; export default queries; -- resolvers/mutations.ts -- import { book } from "~encore/clients"; import { MutationResolvers } from "../__generated__/resolvers-types"; import { APIError } from "encore.dev/api"; // Use the generated `MutationResolvers` type to type check our mutations const mutations: MutationResolvers = { addBook: async (_, { title, author }) => { try { const resp = await book.add({ title, author }); return { book: resp.book, success: true, code: "ok", message: "New book added", }; } catch (err) { const apiError = err as APIError; return { book: null, success: false, code: apiError.code, message: apiError.message, }; } }, }; export default mutations; ``` Now we are ready can create the ApolloServer that makes use of our resolvers and to expose our GraphQL endpoint. 🥐 Still in the `graphql` directory, create a `graphql.ts` file containing: ```ts -- graphql/graphql.ts -- import { api } from "encore.dev/api"; import { ApolloServer, HeaderMap } from "@apollo/server"; import { readFileSync } from "node:fs"; import resolvers from "./resolvers"; import { json } from "node:stream/consumers"; const typeDefs = readFileSync("./schema.graphql", { encoding: "utf-8" }); const server = new ApolloServer({ typeDefs, resolvers, }); await server.start(); export const graphqlAPI = api.raw( { expose: true, path: "/graphql", method: "*" }, async (req, res) => { server.assertStarted("/graphql"); const headers = new HeaderMap(); for (const [key, value] of Object.entries(req.headers)) { if (value !== undefined) { headers.set(key, Array.isArray(value) ? value.join(", ") : value); } } // More on how to use executeHTTPGraphQLRequest: https://www.apollographql.com/docs/apollo-server/integrations/building-integrations/ const httpGraphQLResponse = await server.executeHTTPGraphQLRequest({ httpGraphQLRequest: { headers, method: req.method!.toUpperCase(), body: await json(req), search: new URLSearchParams(req.url ?? "").toString(), }, context: async () => { return { req, res }; }, }); for (const [key, value] of httpGraphQLResponse.headers) { res.setHeader(key, value); } res.statusCode = httpGraphQLResponse.status || 200; if (httpGraphQLResponse.body.kind === "complete") { res.end(httpGraphQLResponse.body.string); return; } for await (const chunk of httpGraphQLResponse.body.asyncIterator) { res.write(chunk); } res.end(); }, ); ``` This creates an [Raw API endpoint](https://encore.dev/docs/ts/primitives/raw-endpoints) available on `/graphql`. In the endpoint we use ApolloServer to handle the GraphQL queries and mutations. We then return the response to the client. If we were to use another GraphQL library other than Apollo, the concept would still be the same: 1. Take client requests with a Raw endpoint. 2. Pass along the request and response objects to the GraphQL library of your choice. 3. Use the library to handle the GraphQL queries and mutations. 4. Return the GraphQL response from the Raw endpoint. ## 6. Trying it out With that, the GraphQL API is done! 🥐Try it out by running `encore run` and opening [https://studio.apollographql.com/sandbox](https://studio.apollographql.com/sandbox) in your browser. Set http://localhost:4000/graphql as your endpoint URL. You should now be able to read the schema and execute queries. Enter the query: ```graphql mutation AddBook { addBook(author: "J.R.R. Tolkien", title: "The Hobbit") { success message code } } ``` Now try the GetBooks query: ```graphql query GetBooks { books { author title } } ``` And you should now see the "The Hobbit" in the list of books. 🥐 Try opening the Local Development Dashboard at [http://localhost:9400](http://localhost:9400) and view the traces that were generated when calling your GraphQL API. ## 7. Deploy ### Self-hosting Encore supports building Docker images directly from the CLI, which can then be self-hosted on your own infrastructure of choice. If your app is using infrastructure resources, such as SQL databases, Pub/Sub, or metrics, you will need to supply a [runtime configuration](https://encore.dev/docs/self-host/configure-infra) your Docker image. 🥐 Build a Docker image by running `encore build docker graphql:v1.0`. This will compile your application using the host machine and then produce a Docker image containing the compiled application. 🥐 Upload the Docker image to the cloud provider of your choice and run it. ### The Encore platform The Encore platform provides automated infrastructure and DevOps. Deploy to a free Encore Cloud development environment, or connect your own AWS or GCP account for production. ### Create account Before deploying with the Encore platform, your app needs to be linked to a free Encore account. If you created the app while logged in, it's already linked, and you can move on to the next step. Otherwise, sign up or log in and register the app by following steps 1 to 3 of [Already have a local app?](https://encore.dev/docs/platform/deploy/deploying#already-have-a-local-app), then continue below. ### Commit changes The final step before you deploy is to commit all changes to the project repo. 🥐 Push your changes and deploy your application to a free Encore Cloud development environment by running: ```shell $ git add -A . $ git commit -m 'Initial commit' $ git push encore ``` Encore will now build and test your app, provision the needed infrastructure, and deploy your application to the cloud. After triggering the deployment, you will see a URL where you can view its progress in the [Encore platform dashboard](https://app.encore.dev). It will look something like: `https://app.encore.dev/$APP_ID/envs/staging/deploys/...` From there you can also see metrics, traces, link your app to a GitHub repo to get automatic deploys on new commits, and connect your own AWS or GCP account to use for production deployment. ### Celebrate with fireworks Now that your app is running in the cloud, let's celebrate with some fireworks: 🥐 In the Cloud Dashboard, open the Command Menu by pressing **Cmd + K** (Mac) or **Ctrl + K** (Windows/Linux). _From here you can easily access all Cloud Dashboard features and for example jump straight to specific services in the Service Catalog or view Traces for specific endpoints._ 🥐 Type `fireworks` in the Command Menu and press enter. Sit back and enjoy the show! ![Fireworks](https://encore.dev/assets/docs/fireworks.jpg) ## Conclusion We've now built a GraphQL API gateway that forwards requests to the application's underlying Encore services in a type-safe way with minimal boilerplate. Note that the concepts discussed here are general and can be easily adapted to any GraphQL schema. Whenever you make a change to the schema or configuration, re-run `npm run generate` to regenerate the resolver types. --- Source: https://encore.dev/docs/ts/frontend/hosting Section: Building (Encore.ts) # Hosting a frontend > Keep using your favorite frontend hosting provider Encore is not opinionated about where you host your frontend, pick the platform that suits your situation best. **Related example:** [encoredev/examples/ts/nextjs-starter](https://github.com/encoredev/examples/tree/main/ts/nextjs-starter) Next.js + Encore TS Web App Starter. Frontend hosted on Vercel, backend on Encore. Create it locally with `encore app create --example=ts/nextjs-starter`. ## Hosting a frontend using Encore Encore is primarily designed for backend development. It is possible to serve a frontend using Encore but for production we recommend that you deploy your frontend using Vercel, Netlify, or a similar service. ### Template engines You can make use of template engines like Handlebars, Pug, or EJS to render HTML files on the server. Learn more about in the [Template Engine](https://encore.dev/docs/ts/frontend/template-engine) docs. ### Serving static assets You can create a `api.static` endpoint that serves static frontend assets, including HTML files. ```ts import { api } from "encore.dev/api"; // Using fallback route to serve all files in the ./assets directory under the root path. export const rootAssets = api.static({ expose: true, path: "/!path", dir: "./assets", // When a file matching the request isn't found, Encore automatically serves a 404 Not Found response. // You can customize the response by setting the notFound option to specify a file that should be served instead: notFound: "./assets/not_found.html", }); ``` Keep in mind that this approach will not work if you have a Single-Page Application (SPA) that uses client-side routing. **Related example:** [encoredev/examples/ts/static-files](https://github.com/encoredev/examples/tree/main/ts/static-files) Static files example, sercing HTML files from a directory. Create it locally with `encore app create --example=ts/static-files`. --- Source: https://encore.dev/docs/ts/frontend/template-engine Section: Building (Encore.ts) # Use a template engine In this guide you will learn how to use a template engine, like [EJS](https://ejs.co) and [Handlebars](https://handlebarsjs.com), to create server-rendered HTML views. **Related example:** [encoredev/examples/ts/template-engine](https://github.com/encoredev/examples/tree/main/ts/template-engine) Using EJS as a template engine with Encore.ts Create it locally with `encore app create --example=ts/template-engine`. ## Serving a specific template file Breakdown of the example: * We import a NPM package for rendering templates, in this case [EJS](https://ejs.co/). * We have a [Raw Endpoint](https://encore.dev/docs/ts/primitives/raw-endpoints) to handle template rendering, in this case we are serving a specific template file (`person.html`) under `/person`. * We make use of the EJS to render the template with the given data. * We set the `content-type` header to `text-html` and then respond with the generated HTML. ```ts -- template/template.ts -- import { api } from "encore.dev/api"; import ejs, { Options } from "ejs"; const BASE_PATH = "./template/views"; const ejsOptions: Options = { views: [BASE_PATH] }; export const serveSpecificTemplate = api.raw( { expose: true, path: "/person", method: "GET" }, async (req, resp) => { const viewPath = `${BASE_PATH}/person.html`; const html = await ejs.renderFile( viewPath, // Supplying data to the view { name: "Simon" }, ejsOptions, ); resp.setHeader("content-type", "text/html"); resp.end(html); }, ); -- template/views/person.html --

Person Page

Name: <%= name %>

``` ## Serving from a dynamic path This example is similar to the one above, but in this case we use a fallback path to serve a template file based on the path. We use the `currentRequest` function to get the `path` and then render the template file based on the `path`. If no path is provided, we default to `index.html`. ```ts import { api } from "encore.dev/api"; import { APICallMeta, currentRequest } from "encore.dev"; import ejs, { Options } from "ejs"; const BASE_PATH = "./template/views"; const ejsOptions: Options = { views: [BASE_PATH] }; export const servePathTemplate = api.raw( { expose: true, path: "/!path", method: "GET" }, async (req, resp) => { const { path } = (currentRequest() as APICallMeta).pathParams; const viewPath = `${BASE_PATH}/${path ?? "index"}.html`; const html = await ejs.renderFile(viewPath, ejsOptions); resp.setHeader("content-type", "text/html"); resp.end(html); }, ); ``` ## Serving inline HTML In this example we are serving inline HTML with EJS. We use the `ejs.render` function to render the inline HTML with the given data. ```ts import { api } from "encore.dev/api"; import ejs, { Options } from "ejs"; const inlineHTML = `

Static Inline HTML Example

Name: <%= name %>!

`; export const serveInlineHTML = api.raw( { expose: true, path: "/html", method: "GET" }, async (req, resp) => { const html = ejs.render(inlineHTML, { name: "Simon" }); resp.setHeader("Content-Type", "text/html"); resp.end(html); }, ); ``` ## Static files In the above example we are fetching a stylesheet from the `/public` path. We can use the `api.static` function to serve all files in the `./assets` directory under the `/public` path prefix: ```ts // Serve all files in the ./assets directory under the /public path prefix. export const assets = api.static({ expose: true, path: "/public/*path", dir: "./assets", }); ``` Learn more about serving static files in the [Static Files](https://encore.dev/docs/ts/primitives/static-assets) guide. --- Source: https://encore.dev/docs/ts/runtime/encore-dev Section: Building (Encore.ts) # encore.dev ## Interfaces ### APICallMeta Describes an API call being processed. #### Properties ##### api `api: APIDesc` Describes the API Endpoint being called. ##### headers `headers: Record` The request headers from the HTTP request. The values are arrays if the header contains multiple values, either separated by ";" or when the header key appears more than once. ##### method `method: Method` The HTTP method used in the API call. ##### middlewareData? `optional middlewareData?: Record` Contains values set in middlewares via `MiddlewareRequest.data`. ##### parsedPayload? `optional parsedPayload?: Record` The parsed request payload, as expected by the application code. Not provided for raw endpoints or when the API endpoint expects no request data. ##### path `path: string` The request URL path used in the API call, excluding any query string parameters. For example "/path/to/endpoint". ##### pathAndQuery `pathAndQuery: string` The request URL path used in the API call, including any query string parameters. For example "/path/to/endpoint?with=querystring". ##### pathParams `pathParams: Record` The parsed path parameters for the API endpoint. The keys are the names of the path parameters, from the API definition. For example \{id: 5\}. ##### type `type: "api-call"` Specifies that the request is an API call. *** ### APIDesc Describes an API endpoint. #### Properties ##### auth `auth: boolean` Whether the endpoint requires auth. ##### endpoint `endpoint: string` The name of the endpoint itself. ##### raw `raw: boolean` Whether the endpoint is a raw endpoint. ##### service `service: string` The name of the service that the endpoint belongs to. ##### tags `tags: string[]` Tags specified on the endpoint. *** ### AppMeta Describes the running Encore application. #### Properties ##### apiBaseUrl `apiBaseUrl: string` The base URL which can be used to call the API of this running application. For local development it is "http://localhost:<port>", typically "http://localhost:4000". If a custom domain is used for this environment it is returned here, but note that changes only take effect at the time of deployment while custom domains can be updated at any time. ##### appId `appId: string` The Encore application ID. If the application is not linked to the Encore platform this will be an empty string. To link to the Encore platform run `encore app link` from your terminal in the root directory of the Encore app. ##### build `build: BuildMeta` Information about the build. ##### deploy `deploy: DeployMeta` Information about the deployment. ##### environment `environment: EnvironmentMeta` Information about the environment the app is running in. *** ### BaseRequestMeta Common fields shared by all request meta types. #### Properties ##### trace? `optional trace?: TraceData` Information about the trace, if the request is being traced *** ### BuildMeta Information about the build that formed the running application. #### Properties ##### revision `revision: string` The git commit that formed the base of this build. ##### uncommittedChanges `uncommittedChanges: boolean` Whether there were uncommitted changes on top of the commit. *** ### DeployMeta Information about the deployment of the running application. #### Properties ##### hostedServices `hostedServices: Record` The services hosted by this deployment, keyed by the service name. ##### id `id: string` The unique id of the deployment. Generated by the Encore Platform. *** ### EnvironmentMeta Describes the environment the Encore application is running in. #### Properties ##### cloud `cloud: CloudProvider` The cloud this is running in. For local development it is "local". ##### name `name: string` The name of environment that this application. For local development it is "local". ##### type `type: EnvironmentType` The type of environment is this application running in. For local development it is "development". *** ### HostedService #### Properties ##### name `name: string` The name of the service *** ### PubSubMessageMeta Describes a Pub/Sub message being processed. #### Properties ##### deliveryAttempt `deliveryAttempt: number` The delivery attempt. The first attempt starts at 1, and increases by 1 for each retry. ##### messageId `messageId: string` The unique id of the Pub/Sub message. It is the same id returned by `topic.publish()`. The message id stays the same across delivery attempts. ##### parsedPayload? `optional parsedPayload?: Record` The parsed request payload, as expected by the application code. ##### service `service: string` The service processing the message. ##### subscription `subscription: string` The name of the Pub/Sub subscription. ##### topic `topic: string` The name of the Pub/Sub topic. ##### type `type: "pubsub-message"` Specifies that the request is a Pub/Sub message. *** ### TraceData Provides information about the active trace. #### Properties ##### extCorrelationId? `optional extCorrelationId?: string` The external correlation id provided when the trace was created, if any. For example via the `Request-Id` or `X-Correlation-Id` headers. ##### parentSpanId? `optional parentSpanId?: string` The span that initiated this span, if any. ##### parentTraceId? `optional parentTraceId?: string` The trace id that initiated this trace, if any. ##### spanId `spanId: string` The current span id. ##### traceId `traceId: string` The trace id. ## Type Aliases ### CloudProvider `type CloudProvider = "aws" | "gcp" | "azure" | "encore" | "local"` Describes what cloud provider the application is running in. *** ### EnvironmentType `type EnvironmentType = "production" | "development" | "ephemeral" | "test"` Describes what type of environment the application is running in. *** ### Method ```ts type Method = | "GET" | "POST" | "PUT" | "PATCH" | "DELETE" | "HEAD" | "OPTIONS" | "CONNECT" | "TRACE"; ``` *** ### RequestMeta `type RequestMeta = APICallMeta | PubSubMessageMeta & BaseRequestMeta` Describes an API call or Pub/Sub message being processed. ## Functions ### appMeta() `function appMeta(): AppMeta` Returns metadata about the running Encore application. The metadata is cached and is the same object each call, and therefore must not be modified by the caller. #### Returns [`AppMeta`](#appmeta) *** ### currentRequest() `function currentRequest(): RequestMeta | undefined` Returns information about the running Encore request, such as API calls and Pub/Sub messages being processed. Returns undefined only if no request is being processed, such as during system initialization. #### Returns [`RequestMeta`](#requestmeta) \| `undefined` --- Source: https://encore.dev/docs/ts/runtime/api Section: Building (Encore.ts) # encore.dev/api ## Enumerations ### ErrCode #### Enumeration Members ##### Aborted `Aborted: "aborted"` Aborted indicates the operation was aborted, typically due to a concurrency issue like sequencer check failures, transaction aborts, etc. See litmus test above for deciding between FailedPrecondition, Aborted, and Unavailable. ##### AlreadyExists `AlreadyExists: "already_exists"` AlreadyExists means an attempt to create an entity failed because one already exists. This error code will not be generated by the gRPC framework. ##### Canceled `Canceled: "canceled"` Canceled indicates the operation was canceled (typically by the caller). Encore will generate this error code when cancellation is requested. ##### DataLoss `DataLoss: "data_loss"` DataLoss indicates unrecoverable data loss or corruption. This error code will not be generated by the gRPC framework. ##### DeadlineExceeded `DeadlineExceeded: "deadline_exceeded"` DeadlineExceeded means operation expired before completion. For operations that change the state of the system, this error may be returned even if the operation has completed successfully. For example, a successful response from a server could have been delayed long enough for the deadline to expire. The gRPC framework will generate this error code when the deadline is exceeded. ##### FailedPrecondition `FailedPrecondition: "failed_precondition"` FailedPrecondition indicates operation was rejected because the system is not in a state required for the operation's execution. For example, directory to be deleted may be non-empty, an rmdir operation is applied to a non-directory, etc. A litmus test that may help a service implementor in deciding between FailedPrecondition, Aborted, and Unavailable: (a) Use Unavailable if the client can retry just the failing call. (b) Use Aborted if the client should retry at a higher-level (e.g., restarting a read-modify-write sequence). (c) Use FailedPrecondition if the client should not retry until the system state has been explicitly fixed. E.g., if an "rmdir" fails because the directory is non-empty, FailedPrecondition should be returned since the client should not retry unless they have first fixed up the directory by deleting files from it. (d) Use FailedPrecondition if the client performs conditional REST Get/Update/Delete on a resource and the resource on the server does not match the condition. E.g., conflicting read-modify-write on the same resource. This error code will not be generated by the gRPC framework. ##### Internal `Internal: "internal"` Internal errors. Means some invariants expected by underlying system has been broken. If you see one of these errors, something is very broken. This error code will be generated by the gRPC framework in several internal error conditions. ##### InvalidArgument `InvalidArgument: "invalid_argument"` InvalidArgument indicates client specified an invalid argument. Note that this differs from FailedPrecondition. It indicates arguments that are problematic regardless of the state of the system (e.g., a malformed file name). This error code will not be generated by the gRPC framework. ##### NotFound `NotFound: "not_found"` NotFound means some requested entity (e.g., file or directory) was not found. This error code will not be generated by the gRPC framework. ##### OK `OK: "ok"` OK indicates the operation was successful. ##### OutOfRange `OutOfRange: "out_of_range"` OutOfRange means operation was attempted past the valid range. E.g., seeking or reading past end of file. Unlike InvalidArgument, this error indicates a problem that may be fixed if the system state changes. For example, a 32-bit file system will generate InvalidArgument if asked to read at an offset that is not in the range [0,2^32-1], but it will generate OutOfRange if asked to read from an offset past the current file size. There is a fair bit of overlap between FailedPrecondition and OutOfRange. We recommend using OutOfRange (the more specific error) when it applies so that callers who are iterating through a space can easily look for an OutOfRange error to detect when they are done. This error code will not be generated by the gRPC framework. ##### PermissionDenied `PermissionDenied: "permission_denied"` PermissionDenied indicates the caller does not have permission to execute the specified operation. It must not be used for rejections caused by exhausting some resource (use ResourceExhausted instead for those errors). It must not be used if the caller cannot be identified (use Unauthenticated instead for those errors). This error code will not be generated by the gRPC core framework, but expect authentication middleware to use it. ##### ResourceExhausted `ResourceExhausted: "resource_exhausted"` ResourceExhausted indicates some resource has been exhausted, perhaps a per-user quota, or perhaps the entire file system is out of space. This error code will be generated by the gRPC framework in out-of-memory and server overload situations, or when a message is larger than the configured maximum size. ##### Unauthenticated `Unauthenticated: "unauthenticated"` Unauthenticated indicates the request does not have valid authentication credentials for the operation. The gRPC framework will generate this error code when the authentication metadata is invalid or a Credentials callback fails, but also expect authentication middleware to generate it. ##### Unavailable `Unavailable: "unavailable"` Unavailable indicates the service is currently unavailable. This is a most likely a transient condition and may be corrected by retrying with a backoff. Note that it is not always safe to retry non-idempotent operations. See litmus test above for deciding between FailedPrecondition, Aborted, and Unavailable. This error code will be generated by the gRPC framework during abrupt shutdown of a server process or network connection. ##### Unimplemented `Unimplemented: "unimplemented"` Unimplemented indicates operation is not implemented or not supported/enabled in this service. This error code will be generated by the gRPC framework. Most commonly, you will see this error code when a method implementation is missing on the server. It can also be generated for unknown compression algorithms or a disagreement as to whether an RPC should be streaming. ##### Unknown `Unknown: "unknown"` Unknown error. An example of where this error may be returned is if a Status value received from another address space belongs to an error-space that is not known in this address space. Also errors raised by APIs that do not return enough error information may be converted to this error. Encore will generate this error code in the above two mentioned cases. ## Classes ### APIError #### Extends - `Error` #### Constructors ##### Constructor ```ts new APIError( code, msg, cause?, details?): APIError; ``` Constructs an APIError with the given error code, message, and (optionally) cause. ###### Parameters ###### code [`ErrCode`](#errcode) ###### msg `string` ###### cause? `Error` ###### details? [`ErrDetails`](#errdetails) ###### Returns [`APIError`](#apierror) ###### Overrides `Error.constructor` #### Properties ##### code `readonly code: ErrCode` The error code. ##### details? `readonly optional details?: ErrDetails` #### Methods ##### withDetails() `withDetails(details): APIError` Constructs a new APIError from the previous one with the provided details ###### Parameters ###### details [`ErrDetails`](#errdetails) ###### Returns [`APIError`](#apierror) ##### aborted() `static aborted(msg, cause?): APIError` Constructs an APIError with the Aborted error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### alreadyExists() `static alreadyExists(msg, cause?): APIError` Constructs an APIError with the AlreadyExists error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### canceled() `static canceled(msg, cause?): APIError` Constructs an APIError with the Canceled error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### dataLoss() `static dataLoss(msg, cause?): APIError` Constructs an APIError with the DataLoss error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### deadlineExceeded() `static deadlineExceeded(msg, cause?): APIError` Constructs an APIError with the DeadlineExceeded error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### failedPrecondition() `static failedPrecondition(msg, cause?): APIError` Constructs an APIError with the FailedPrecondition error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### internal() `static internal(msg, cause?): APIError` Constructs an APIError with the Internal error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### invalidArgument() `static invalidArgument(msg, cause?): APIError` Constructs an APIError with the InvalidArgument error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### notFound() `static notFound(msg, cause?): APIError` Constructs an APIError with the NotFound error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### outOfRange() `static outOfRange(msg, cause?): APIError` Constructs an APIError with the OutOfRange error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### permissionDenied() `static permissionDenied(msg, cause?): APIError` Constructs an APIError with the PermissionDenied error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### resourceExhausted() `static resourceExhausted(msg, cause?): APIError` Constructs an APIError with the ResourceExhausted error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### unauthenticated() `static unauthenticated(msg, cause?): APIError` Constructs an APIError with the Unauthenticated error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### unavailable() `static unavailable(msg, cause?): APIError` Constructs an APIError with the Unavailable error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### unimplemented() `static unimplemented(msg, cause?): APIError` Constructs an APIError with the Unimplemented error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) ##### unknown() `static unknown(msg, cause?): APIError` Constructs an APIError with the Unknown error code. ###### Parameters ###### msg `string` ###### cause? `Error` ###### Returns [`APIError`](#apierror) *** ### Gateway #### Constructors ##### Constructor `new Gateway(cfg): Gateway` ###### Parameters ###### cfg [`GatewayConfig`](#gatewayconfig) ###### Returns [`Gateway`](#gateway) #### Properties ##### cfg `readonly cfg: GatewayConfig` ##### name `readonly name: string` *** ### HandlerResponse #### Constructors ##### Constructor `new HandlerResponse(payload): HandlerResponse` ###### Parameters ###### payload `any` ###### Returns [`HandlerResponse`](#handlerresponse) #### Properties ##### payload `payload: any` The payload returned by the handler when the handler is either a typed handler or stream handler. #### Accessors ##### header ###### Get Signature `get header(): ResponseHeader` header can be used by middlewares to set headers to the response. This only works for typed handler. For raw handlers see MiddlewareRequest.rawResponse. ###### Returns [`ResponseHeader`](#responseheader) ##### status ###### Set Signature `set status(s): void` Override the http status code for successful requests for typed endpoints. ###### Parameters ###### s `number` ###### Returns `void` *** ### IterableSocket #### Constructors ##### Constructor `new IterableSocket(socket): IterableSocket` ###### Parameters ###### socket `Socket` ###### Returns [`IterableSocket`](#iterablesocket) #### Methods ##### \[asyncIterator\]() `asyncIterator: AsyncGenerator` ###### Returns `AsyncGenerator`\<`any`, `void`, `unknown`\> ##### close() `close(): void` ###### Returns `void` ##### recv() `recv(): Promise>` ###### Returns `Promise`\<`Record`\<`string`, `any`\>\> ##### send() `send(msg): void` ###### Parameters ###### msg `Record`\<`string`, `any`\> ###### Returns `void` *** ### IterableStream #### Constructors ##### Constructor `new IterableStream(stream): IterableStream` ###### Parameters ###### stream `Stream` ###### Returns [`IterableStream`](#iterablestream) #### Methods ##### \[asyncIterator\]() `asyncIterator: AsyncGenerator` ###### Returns `AsyncGenerator`\<`any`, `void`, `unknown`\> ##### recv() `recv(): Promise>` ###### Returns `Promise`\<`Record`\<`string`, `any`\>\> *** ### MiddlewareRequest #### Constructors ##### Constructor ```ts new MiddlewareRequest( stream?, rawReq?, rawResp?): MiddlewareRequest; ``` ###### Parameters ###### stream? \| [`IterableStream`](#iterablestream) \| [`IterableSocket`](#iterablesocket) \| [`Sink`](#sink) ###### rawReq? [`RawRequest`](#rawrequest-1) ###### rawResp? [`RawResponse`](#rawresponse-1) ###### Returns [`MiddlewareRequest`](#middlewarerequest) #### Accessors ##### data ###### Get Signature `get data(): Record` data can be used to pass data from middlewares to the handler. The data will be available via `currentRequest()` ###### Returns `Record`\<`string`, `any`\> ##### rawRequest ###### Get Signature `get rawRequest(): RawRequest | undefined` rawRequest is set when the handler is a raw request handler. The returned value is a Node.js `http.IncomingMessage`. ###### Returns [`RawRequest`](#rawrequest-1) \| `undefined` ##### rawResponse ###### Get Signature `get rawResponse(): RawResponse | undefined` rawResponse is set when the handler is a raw request handler. The returned value is a Node.js `http.ServerResponse`. ###### Returns [`RawResponse`](#rawresponse-1) \| `undefined` ##### requestMeta ###### Get Signature `get requestMeta(): RequestMeta | undefined` requestMeta is set when the handler is a typed handler or a stream handler. for raw handlers, see rawRequest and rawResponse. ###### Returns [`RequestMeta`](https://encore.dev/ts/runtime/mod#requestmeta) \| `undefined` ##### stream ###### Get Signature ```ts get stream(): | IterableStream | IterableSocket | Sink | undefined; ``` stream is set when the handler is a stream handler. ###### Returns \| [`IterableStream`](#iterablestream) \| [`IterableSocket`](#iterablesocket) \| [`Sink`](#sink) \| `undefined` *** ### RawRequest #### Extends - `Readable` #### Constructors ##### Constructor `new RawRequest(req, body): RawRequest` ###### Parameters ###### req `Request` ###### body `BodyReader` ###### Returns [`RawRequest`](#rawrequest-1) ###### Overrides `stream.Readable.constructor` #### Properties ##### \_headersDistinct `_headersDistinct: Dict | undefined` ##### \_rawHeaders `_rawHeaders: string[] | undefined` ##### \_url `_url: string | undefined` ##### complete `complete: boolean` ##### connection `readonly connection: Socket | null` ##### rawTrailers `rawTrailers: string[]` ##### socket `readonly socket: Socket | null` ##### trailers `trailers: Dict` ##### trailersDistinct `trailersDistinct: Dict` #### Accessors ##### headers ###### Get Signature `get headers(): IncomingHttpHeaders` ###### Returns `IncomingHttpHeaders` ##### headersDistinct ###### Get Signature `get headersDistinct(): Dict` ###### Returns `Dict`\<`string`[]\> ##### method ###### Get Signature `get method(): string` ###### Returns `string` ##### rawHeaders ###### Get Signature `get rawHeaders(): string[]` ###### Returns `string`[] ##### url ###### Get Signature `get url(): string` ###### Returns `string` ###### Set Signature `set url(value): void` ###### Parameters ###### value `string` ###### Returns `void` #### Methods ##### \_read() `_read(size): void` ###### Parameters ###### size `number` ###### Returns `void` ###### Overrides `stream.Readable._read` ##### setTimeout() `setTimeout(msecs, callback?): this` ###### Parameters ###### msecs `number` ###### callback? () => `void` ###### Returns `this` *** ### RawResponse #### Extends - `Writable` #### Constructors ##### Constructor `new RawResponse(req, w): RawResponse` ###### Parameters ###### req [`RawRequest`](#rawrequest-1) ###### w `ResponseWriter` ###### Returns [`RawResponse`](#rawresponse-1) ###### Overrides `stream.Writable.constructor` #### Properties ##### chunkedEncoding `chunkedEncoding: boolean` ##### connection `readonly connection: Socket | null` ##### finished `finished: boolean` ##### headersSent `headersSent: boolean` ##### req `readonly req: RawRequest` ##### sendDate `sendDate: boolean` ##### shouldKeepAlive `shouldKeepAlive: boolean` ##### socket `readonly socket: Socket | null` ##### statusCode `statusCode: number` ##### statusMessage `statusMessage: string | undefined` ##### strictContentLength `strictContentLength: boolean` #### Methods ##### \_final() `_final(callback): void` ###### Parameters ###### callback (`error?`) => `void` ###### Returns `void` ###### Overrides `stream.Writable._final` ##### \_implicitHeader() `_implicitHeader(): void` ###### Returns `void` ##### \_write() ```ts _write( chunk, _encoding, callback): void; ``` ###### Parameters ###### chunk `Buffer` ###### \_encoding `BufferEncoding` ###### callback (`error?`) => `void` ###### Returns `void` ###### Overrides `stream.Writable._write` ##### \_writeHeaderIfNeeded() `_writeHeaderIfNeeded(): void` ###### Returns `void` ##### \_writev() `_writev(chunks, callback): void` ###### Parameters ###### chunks \{ `chunk`: `Buffer`; \}[] ###### callback (`error?`) => `void` ###### Returns `void` ###### Overrides `stream.Writable._writev` ##### addTrailers() `addTrailers(headers): void` ###### Parameters ###### headers `OutgoingHttpHeaders` \| readonly \[`string`, `string`\][] ###### Returns `void` ##### appendHeader() `appendHeader(name, value): this` ###### Parameters ###### name `string` ###### value `string` \| `number` \| `string`[] ###### Returns `this` ##### flushHeaders() `flushHeaders(): void` ###### Returns `void` ##### getHeader() `getHeader(name): string | number | string[] | undefined` ###### Parameters ###### name `string` ###### Returns `string` \| `number` \| `string`[] \| `undefined` ##### getHeaderNames() `getHeaderNames(): string[]` ###### Returns `string`[] ##### getHeaders() `getHeaders(): OutgoingHttpHeaders` ###### Returns `OutgoingHttpHeaders` ##### hasHeader() `hasHeader(name): boolean` ###### Parameters ###### name `string` ###### Returns `boolean` ##### removeHeader() `removeHeader(name): void` ###### Parameters ###### name `string` ###### Returns `void` ##### setHeader() `setHeader(name, value): this` ###### Parameters ###### name `string` ###### value `string` \| `number` \| `string`[] ###### Returns `this` ##### setTimeout() `setTimeout(msecs, callback?): this` ###### Parameters ###### msecs `number` ###### callback? () => `void` ###### Returns `this` ##### write() ###### Call Signature `write(chunk, callback?): boolean` The `writable.write()` method writes some data to the stream, and calls the supplied `callback` once the data has been fully handled. If an error occurs, the `callback` will be called with the error as its first argument. The `callback` is called asynchronously and before `'error'` is emitted. The return value is `true` if the internal buffer is less than the`highWaterMark` configured when the stream was created after admitting `chunk`. If `false` is returned, further attempts to write data to the stream should stop until the `'drain'` event is emitted. While a stream is not draining, calls to `write()` will buffer `chunk`, and return false. Once all currently buffered chunks are drained (accepted for delivery by the operating system), the `'drain'` event will be emitted. Once `write()` returns false, do not write more chunks until the `'drain'` event is emitted. While calling `write()` on a stream that is not draining is allowed, Node.js will buffer all written chunks until maximum memory usage occurs, at which point it will abort unconditionally. Even before it aborts, high memory usage will cause poor garbage collector performance and high RSS (which is not typically released back to the system, even after the memory is no longer required). Since TCP sockets may never drain if the remote peer does not read the data, writing a socket that is not draining may lead to a remotely exploitable vulnerability. Writing data while the stream is not draining is particularly problematic for a `Transform`, because the `Transform` streams are paused by default until they are piped or a `'data'` or `'readable'` event handler is added. If the data to be written can be generated or fetched on demand, it is recommended to encapsulate the logic into a `Readable` and use [pipe](https://nodejs.org/api/stream.html#readablepipedestination-options). However, if calling `write()` is preferred, it is possible to respect backpressure and avoid memory issues using the `'drain'` event: ```js function write(data, cb) { if (!stream.write(data)) { stream.once('drain', cb); } else { process.nextTick(cb); } } // Wait for cb to be called before doing any other write. write('hello', () => { console.log('Write completed, do more writes now.'); }); ``` A `Writable` stream in object mode will always ignore the `encoding` argument. ###### Parameters ###### chunk `any` Optional data to write. For streams not operating in object mode, `chunk` must be a string, `Buffer` or `Uint8Array`. For object mode streams, `chunk` may be any JavaScript value other than `null`. ###### callback? (`error`) => `void` Callback for when this chunk of data is flushed. ###### Returns `boolean` `false` if the stream wishes for the calling code to wait for the `'drain'` event to be emitted before continuing to write additional data; otherwise `true`. ###### Since v0.9.4 ###### Overrides `stream.Writable.write` ###### Call Signature ```ts write( chunk, encoding, callback?): boolean; ``` ###### Parameters ###### chunk `any` ###### encoding `BufferEncoding` ###### callback? (`error`) => `void` ###### Returns `boolean` ###### Overrides `stream.Writable.write` ##### writeHead() ###### Call Signature `writeHead(statusCode, headers?): this` ###### Parameters ###### statusCode `number` ###### headers? `OutgoingHttpHeaders` \| `OutgoingHttpHeader`[] ###### Returns `this` ###### Call Signature ```ts writeHead( statusCode, statusMessage?, headers?): this; ``` ###### Parameters ###### statusCode `number` ###### statusMessage? `string` ###### headers? `OutgoingHttpHeaders` \| `OutgoingHttpHeader`[] ###### Returns `this` *** ### ResponseHeader #### Constructors ##### Constructor `new ResponseHeader(): ResponseHeader` ###### Returns [`ResponseHeader`](#responseheader) #### Properties ##### headers `headers: Record` #### Methods ##### add() `add(key, value): void` add adds a header value to a key, if a previous middleware has already set a value, they will be appended. ###### Parameters ###### key `string` ###### value `string` \| `string`[] ###### Returns `void` ##### set() `set(key, value): void` set will set a header value for a key, if a previous middleware has already set a value, it will be overridden. ###### Parameters ###### key `string` ###### value `string` \| `string`[] ###### Returns `void` *** ### Sink #### Constructors ##### Constructor `new Sink(sink): Sink` ###### Parameters ###### sink `Sink` ###### Returns [`Sink`](#sink) #### Methods ##### close() `close(): void` ###### Returns `void` ##### send() `send(msg): void` ###### Parameters ###### msg `Record`\<`string`, `any`\> ###### Returns `void` *** ### StaticAssets #### Constructors ##### Constructor `new StaticAssets(options): StaticAssets` ###### Parameters ###### options [`StaticOptions`](#staticoptions) ###### Returns [`StaticAssets`](#staticassets) #### Properties ##### options `readonly options: StaticOptions` ## Interfaces ### APIOptions #### Properties ##### auth? `optional auth?: boolean` Whether or not the request must contain valid authentication credentials. If set to true and the request is not authenticated, Encore returns a 401 Unauthorized error. Defaults to false if not specified. ##### bodyLimit? `optional bodyLimit?: number | null` The maximum body size, in bytes. If the request body exceeds this value, Encore stops request processing and returns an error. If left unspecified it defaults to a reasonable default (currently 2MiB). If set to `null`, the body size is unlimited. ##### expose? `optional expose?: boolean` Whether or not to make this endpoint publicly accessible. If false, the endpoint is only accessible from the internal network. Defaults to false if not specified. ##### method? `optional method?: Method | Method[] | "*"` The HTTP method(s) to match for this endpoint. Use "*" to match any method. ##### path? `optional path?: string` The request path to match for this endpoint. Use `:` to define single-segment parameters, e.g. `/users/:id`. Use `*` to match any number of segments, e.g. `/files/*path`. If not specified, it defaults to `/.`. ##### sensitive? `optional sensitive?: boolean` When set to true, request information such as payloads and headers will be excluded from traces. ##### tags? `optional tags?: string[]` Tags to filter endpoints when generating clients and in middlewares. *** ### CallOpts Options when making api calls. This interface will be extended with additional fields from app's generated code. *** ### GatewayConfig #### Properties ##### authHandler? `optional authHandler?: AuthHandlerBrand` *** ### Middleware() #### Extends - [`MiddlewareFn`](#middlewarefn) `Middleware(req, next): Promise` #### Parameters ##### req [`MiddlewareRequest`](#middlewarerequest) ##### next [`Next`](#next) #### Returns `Promise`\<[`HandlerResponse`](#handlerresponse)\> #### Properties ##### options? `optional options?: MiddlewareOptions` *** ### MiddlewareOptions #### Properties ##### target? ```ts optional target?: { auth?: boolean; expose?: boolean; isRaw?: boolean; isStream?: boolean; tags?: string[]; }; ``` Configuration for what endpoints that should be targeted by the middleware ###### auth? `optional auth?: boolean` If set, only run middleware on endpoints that either require or not requires auth. ###### expose? `optional expose?: boolean` If set, only run middleware on endpoints that are either exposed or not exposed. ###### isRaw? `optional isRaw?: boolean` If set, only run middleware on endpoints that are raw endpoints. ###### isStream? `optional isStream?: boolean` If set, only run middleware on endpoints that are stream endpoints. ###### tags? `optional tags?: string[]` If set, only run middleware on endpoints that have specific tags. These tags are evaluated with OR, meaning the middleware applies to an API if the API has at least one of those tags. *** ### StaticOptions #### Properties ##### auth? `optional auth?: boolean` Whether or not the request must contain valid authentication credentials. If set to true and the request is not authenticated, Encore returns a 401 Unauthorized error. Defaults to false if not specified. ##### dir `dir: string` The relative path to the directory containing the static files to serve. The provided path must be a subdirectory from the calling file's directory. ##### expose? `optional expose?: boolean` Whether or not to make this endpoint publicly accessible. If false, the endpoint is only accessible from the internal network. Defaults to false if not specified. ##### headers? `optional headers?: Record` Custom HTTP headers to apply to all static files served. ###### Example ```typescript headers: { "Cache-Control": "public, max-age=3600", "X-Content-Type-Options": "nosniff", } ``` ##### notFound? `optional notFound?: string` Path to the file to serve when the requested file is not found. The path must be a relative path to within the calling file's directory. ##### notFoundStatus? `optional notFoundStatus?: number` Http Status code used when serving notFound fallback. Defaults to 404. ##### path? `optional path?: string` The request path to match for this endpoint. Use `:` to define single-segment parameters, e.g. `/users/:id`. Use `*` to match any number of segments, e.g. `/files/*path`. If not specified, it defaults to `/.`. *** ### StreamIn #### Extends - `AsyncIterable`\<`Request`\> #### Type Parameters ##### Request `Request` #### Properties ##### recv `recv: () => Promise` ###### Returns `Promise`\<`Request`\> *** ### StreamOptions #### Properties ##### auth? `optional auth?: boolean` Whether or not the request must contain valid authentication credentials. If set to true and the request is not authenticated, Encore returns a 401 Unauthorized error. Defaults to false if not specified. ##### expose? `optional expose?: boolean` Whether or not to make this endpoint publicly accessible. If false, the endpoint is only accessible from the internal network. Defaults to false if not specified. ##### path? `optional path?: string` The request path to match for this endpoint. Use `:` to define single-segment parameters, e.g. `/users/:id`. Use `*` to match any number of segments, e.g. `/files/*path`. If not specified, it defaults to `/.`. ##### sensitive? `optional sensitive?: boolean` When set to true, request information such as payloads and headers will be excluded from traces. ##### tags? `optional tags?: string[]` Tags to filter endpoints when generating clients and in middlewares. *** ### StreamOut #### Extended by - [`StreamOutWithResponse`](#streamoutwithresponse) #### Type Parameters ##### Response `Response` #### Properties ##### close `close: () => Promise` ###### Returns `Promise`\<`void`\> ##### send `send: (msg) => Promise` ###### Parameters ###### msg `Response` ###### Returns `Promise`\<`void`\> *** ### StreamOutWithResponse #### Extends - [`StreamOut`](#streamout)\<`Request`\> #### Type Parameters ##### Request `Request` ##### Response `Response` #### Properties ##### close `close: () => Promise` ###### Returns `Promise`\<`void`\> ###### Inherited from [`StreamOut`](#streamout).[`close`](#close-2) ##### response `response: () => Promise` ###### Returns `Promise`\<`Response`\> ##### send `send: (msg) => Promise` ###### Parameters ###### msg `Request` ###### Returns `Promise`\<`void`\> ###### Inherited from [`StreamOut`](#streamout).[`send`](#send-2) ## Type Aliases ### Cookie `type Cookie = TypeOrName extends string ? CookieWithOptions : CookieWithOptions` #### Type Parameters ##### TypeOrName `TypeOrName` *extends* `string` \| `number` \| `boolean` \| `Date` = `string` ##### Name `Name` *extends* `string` = `""` *** ### CookieWithOptions ```ts type CookieWithOptions = { domain?: string; expires?: Date; httpOnly?: boolean; maxAge?: number; partitioned?: boolean; path?: string; sameSite?: "Strict" | "Lax" | "None"; secure?: boolean; value: T; }; ``` #### Type Parameters ##### T `T` #### Properties ##### domain? `optional domain?: string` ##### expires? `optional expires?: Date` ##### httpOnly? `optional httpOnly?: boolean` ##### maxAge? `optional maxAge?: number` ##### partitioned? `optional partitioned?: boolean` ##### path? `optional path?: string` ##### sameSite? `optional sameSite?: "Strict" | "Lax" | "None"` ##### secure? `optional secure?: boolean` ##### value `value: T` *** ### ErrDetails `type ErrDetails = Record` Additional structured error details that can be attached to an APIError. *** ### Header `type Header = TypeOrName extends string ? string : TypeOrName` #### Type Parameters ##### TypeOrName `TypeOrName` *extends* `string` \| `number` \| `boolean` \| `Date` = `string` ##### Name `Name` *extends* `string` = `""` *** ### HttpStatus `type HttpStatus = typeof HttpStatusValues[keyof typeof HttpStatusValues]` The union of all HTTP status code numeric values. *** ### Method ```ts type Method = | "GET" | "POST" | "PUT" | "DELETE" | "PATCH" | "HEAD" | "OPTIONS" | "TRACE" | "CONNECT"; ``` *** ### MiddlewareFn `type MiddlewareFn = (req, next) => Promise` #### Parameters ##### req [`MiddlewareRequest`](#middlewarerequest) ##### next [`Next`](#next) #### Returns `Promise`\<[`HandlerResponse`](#handlerresponse)\> *** ### Next `type Next = (req) => Promise` #### Parameters ##### req [`MiddlewareRequest`](#middlewarerequest) #### Returns `Promise`\<[`HandlerResponse`](#handlerresponse)\> *** ### Query `type Query = TypeOrName extends string ? string : TypeOrName` #### Type Parameters ##### TypeOrName `TypeOrName` *extends* `string` \| `string`[] \| `number` \| `number`[] \| `boolean` \| `boolean`[] \| `Date` \| `Date`[] = `string` ##### Name `Name` *extends* `string` = `""` *** ### RawHandler `type RawHandler = (req, resp) => void` #### Parameters ##### req `IncomingMessage` ##### resp `ServerResponse` #### Returns `void` *** ### StreamInHandlerFn `type StreamInHandlerFn = HandshakeData extends void ? (stream) => Promise : (data, stream) => Promise` #### Type Parameters ##### HandshakeData `HandshakeData` ##### Request `Request` ##### Response `Response` *** ### StreamInOut `type StreamInOut = StreamIn & StreamOut` #### Type Parameters ##### Request `Request` ##### Response `Response` *** ### StreamInOutHandlerFn `type StreamInOutHandlerFn = HandshakeData extends void ? (stream) => Promise : (data, stream) => Promise` #### Type Parameters ##### HandshakeData `HandshakeData` ##### Request `Request` ##### Response `Response` *** ### StreamOutHandlerFn `type StreamOutHandlerFn = HandshakeData extends void ? (stream) => Promise : (data, stream) => Promise` #### Type Parameters ##### HandshakeData `HandshakeData` ##### Response `Response` ## Variables ### HttpStatus ```ts const HttpStatus: { Accepted: 202; AlreadyReported: 208; BadGateway: 502; BadRequest: 400; Conflict: 409; Continue: 100; Created: 201; EarlyHints: 103; ExpectationFailed: 417; FailedDependency: 424; Forbidden: 403; Found: 302; GatewayTimeout: 504; Gone: 410; HTTPVersionNotSupported: 505; ImATeapot: 418; IMUsed: 226; InsufficientStorage: 507; InternalServerError: 500; LengthRequired: 411; Locked: 423; LoopDetected: 508; MethodNotAllowed: 405; MisdirectedRequest: 421; MovedPermanently: 301; MultipleChoices: 300; MultiStatus: 207; NetworkAuthenticationRequired: 511; NoContent: 204; NonAuthoritativeInformation: 203; NotAcceptable: 406; NotExtended: 510; NotFound: 404; NotImplemented: 501; NotModified: 304; OK: 200; PartialContent: 206; PayloadTooLarge: 413; PaymentRequired: 402; PermanentRedirect: 308; PreconditionFailed: 412; PreconditionRequired: 428; Processing: 102; ProxyAuthenticationRequired: 407; RangeNotSatisfiable: 416; RequestHeaderFieldsTooLarge: 431; RequestTimeout: 408; ResetContent: 205; SeeOther: 303; ServiceUnavailable: 503; SwitchingProtocols: 101; SwitchProxy: 306; TemporaryRedirect: 307; TooEarly: 425; TooManyRequests: 429; Unauthorized: 401; UnavailableForLegalReasons: 451; UnprocessableEntity: 422; UnsupportedMediaType: 415; UpgradeRequired: 426; URITooLong: 414; UseProxy: 305; VariantAlsoNegotiates: 506; } = HttpStatusValues; ``` A map of HTTP status code names to their numeric values. #### Type Declaration ##### Accepted `readonly Accepted: 202 = 202` ##### AlreadyReported `readonly AlreadyReported: 208 = 208` ##### BadGateway `readonly BadGateway: 502 = 502` ##### BadRequest `readonly BadRequest: 400 = 400` ##### Conflict `readonly Conflict: 409 = 409` ##### Continue `readonly Continue: 100 = 100` ##### Created `readonly Created: 201 = 201` ##### EarlyHints `readonly EarlyHints: 103 = 103` ##### ExpectationFailed `readonly ExpectationFailed: 417 = 417` ##### FailedDependency `readonly FailedDependency: 424 = 424` ##### Forbidden `readonly Forbidden: 403 = 403` ##### Found `readonly Found: 302 = 302` ##### GatewayTimeout `readonly GatewayTimeout: 504 = 504` ##### Gone `readonly Gone: 410 = 410` ##### HTTPVersionNotSupported `readonly HTTPVersionNotSupported: 505 = 505` ##### ImATeapot `readonly ImATeapot: 418 = 418` ##### IMUsed `readonly IMUsed: 226 = 226` ##### InsufficientStorage `readonly InsufficientStorage: 507 = 507` ##### InternalServerError `readonly InternalServerError: 500 = 500` ##### LengthRequired `readonly LengthRequired: 411 = 411` ##### Locked `readonly Locked: 423 = 423` ##### LoopDetected `readonly LoopDetected: 508 = 508` ##### MethodNotAllowed `readonly MethodNotAllowed: 405 = 405` ##### MisdirectedRequest `readonly MisdirectedRequest: 421 = 421` ##### MovedPermanently `readonly MovedPermanently: 301 = 301` ##### MultipleChoices `readonly MultipleChoices: 300 = 300` ##### MultiStatus `readonly MultiStatus: 207 = 207` ##### NetworkAuthenticationRequired `readonly NetworkAuthenticationRequired: 511 = 511` ##### NoContent `readonly NoContent: 204 = 204` ##### NonAuthoritativeInformation `readonly NonAuthoritativeInformation: 203 = 203` ##### NotAcceptable `readonly NotAcceptable: 406 = 406` ##### NotExtended `readonly NotExtended: 510 = 510` ##### NotFound `readonly NotFound: 404 = 404` ##### NotImplemented `readonly NotImplemented: 501 = 501` ##### NotModified `readonly NotModified: 304 = 304` ##### OK `readonly OK: 200 = 200` ##### PartialContent `readonly PartialContent: 206 = 206` ##### PayloadTooLarge `readonly PayloadTooLarge: 413 = 413` ##### PaymentRequired `readonly PaymentRequired: 402 = 402` ##### PermanentRedirect `readonly PermanentRedirect: 308 = 308` ##### PreconditionFailed `readonly PreconditionFailed: 412 = 412` ##### PreconditionRequired `readonly PreconditionRequired: 428 = 428` ##### Processing `readonly Processing: 102 = 102` ##### ProxyAuthenticationRequired `readonly ProxyAuthenticationRequired: 407 = 407` ##### RangeNotSatisfiable `readonly RangeNotSatisfiable: 416 = 416` ##### RequestHeaderFieldsTooLarge `readonly RequestHeaderFieldsTooLarge: 431 = 431` ##### RequestTimeout `readonly RequestTimeout: 408 = 408` ##### ResetContent `readonly ResetContent: 205 = 205` ##### SeeOther `readonly SeeOther: 303 = 303` ##### ServiceUnavailable `readonly ServiceUnavailable: 503 = 503` ##### SwitchingProtocols `readonly SwitchingProtocols: 101 = 101` ##### SwitchProxy `readonly SwitchProxy: 306 = 306` ##### TemporaryRedirect `readonly TemporaryRedirect: 307 = 307` ##### TooEarly `readonly TooEarly: 425 = 425` ##### TooManyRequests `readonly TooManyRequests: 429 = 429` ##### Unauthorized `readonly Unauthorized: 401 = 401` ##### UnavailableForLegalReasons `readonly UnavailableForLegalReasons: 451 = 451` ##### UnprocessableEntity `readonly UnprocessableEntity: 422 = 422` ##### UnsupportedMediaType `readonly UnsupportedMediaType: 415 = 415` ##### UpgradeRequired `readonly UpgradeRequired: 426 = 426` ##### URITooLong `readonly URITooLong: 414 = 414` ##### UseProxy `readonly UseProxy: 305 = 305` ##### VariantAlsoNegotiates `readonly VariantAlsoNegotiates: 506 = 506` *** ### HttpStatusValues ```ts const HttpStatusValues: { Accepted: 202; AlreadyReported: 208; BadGateway: 502; BadRequest: 400; Conflict: 409; Continue: 100; Created: 201; EarlyHints: 103; ExpectationFailed: 417; FailedDependency: 424; Forbidden: 403; Found: 302; GatewayTimeout: 504; Gone: 410; HTTPVersionNotSupported: 505; ImATeapot: 418; IMUsed: 226; InsufficientStorage: 507; InternalServerError: 500; LengthRequired: 411; Locked: 423; LoopDetected: 508; MethodNotAllowed: 405; MisdirectedRequest: 421; MovedPermanently: 301; MultipleChoices: 300; MultiStatus: 207; NetworkAuthenticationRequired: 511; NoContent: 204; NonAuthoritativeInformation: 203; NotAcceptable: 406; NotExtended: 510; NotFound: 404; NotImplemented: 501; NotModified: 304; OK: 200; PartialContent: 206; PayloadTooLarge: 413; PaymentRequired: 402; PermanentRedirect: 308; PreconditionFailed: 412; PreconditionRequired: 428; Processing: 102; ProxyAuthenticationRequired: 407; RangeNotSatisfiable: 416; RequestHeaderFieldsTooLarge: 431; RequestTimeout: 408; ResetContent: 205; SeeOther: 303; ServiceUnavailable: 503; SwitchingProtocols: 101; SwitchProxy: 306; TemporaryRedirect: 307; TooEarly: 425; TooManyRequests: 429; Unauthorized: 401; UnavailableForLegalReasons: 451; UnprocessableEntity: 422; UnsupportedMediaType: 415; UpgradeRequired: 426; URITooLong: 414; UseProxy: 305; VariantAlsoNegotiates: 506; }; ``` #### Type Declaration ##### Accepted `readonly Accepted: 202 = 202` ##### AlreadyReported `readonly AlreadyReported: 208 = 208` ##### BadGateway `readonly BadGateway: 502 = 502` ##### BadRequest `readonly BadRequest: 400 = 400` ##### Conflict `readonly Conflict: 409 = 409` ##### Continue `readonly Continue: 100 = 100` ##### Created `readonly Created: 201 = 201` ##### EarlyHints `readonly EarlyHints: 103 = 103` ##### ExpectationFailed `readonly ExpectationFailed: 417 = 417` ##### FailedDependency `readonly FailedDependency: 424 = 424` ##### Forbidden `readonly Forbidden: 403 = 403` ##### Found `readonly Found: 302 = 302` ##### GatewayTimeout `readonly GatewayTimeout: 504 = 504` ##### Gone `readonly Gone: 410 = 410` ##### HTTPVersionNotSupported `readonly HTTPVersionNotSupported: 505 = 505` ##### ImATeapot `readonly ImATeapot: 418 = 418` ##### IMUsed `readonly IMUsed: 226 = 226` ##### InsufficientStorage `readonly InsufficientStorage: 507 = 507` ##### InternalServerError `readonly InternalServerError: 500 = 500` ##### LengthRequired `readonly LengthRequired: 411 = 411` ##### Locked `readonly Locked: 423 = 423` ##### LoopDetected `readonly LoopDetected: 508 = 508` ##### MethodNotAllowed `readonly MethodNotAllowed: 405 = 405` ##### MisdirectedRequest `readonly MisdirectedRequest: 421 = 421` ##### MovedPermanently `readonly MovedPermanently: 301 = 301` ##### MultipleChoices `readonly MultipleChoices: 300 = 300` ##### MultiStatus `readonly MultiStatus: 207 = 207` ##### NetworkAuthenticationRequired `readonly NetworkAuthenticationRequired: 511 = 511` ##### NoContent `readonly NoContent: 204 = 204` ##### NonAuthoritativeInformation `readonly NonAuthoritativeInformation: 203 = 203` ##### NotAcceptable `readonly NotAcceptable: 406 = 406` ##### NotExtended `readonly NotExtended: 510 = 510` ##### NotFound `readonly NotFound: 404 = 404` ##### NotImplemented `readonly NotImplemented: 501 = 501` ##### NotModified `readonly NotModified: 304 = 304` ##### OK `readonly OK: 200 = 200` ##### PartialContent `readonly PartialContent: 206 = 206` ##### PayloadTooLarge `readonly PayloadTooLarge: 413 = 413` ##### PaymentRequired `readonly PaymentRequired: 402 = 402` ##### PermanentRedirect `readonly PermanentRedirect: 308 = 308` ##### PreconditionFailed `readonly PreconditionFailed: 412 = 412` ##### PreconditionRequired `readonly PreconditionRequired: 428 = 428` ##### Processing `readonly Processing: 102 = 102` ##### ProxyAuthenticationRequired `readonly ProxyAuthenticationRequired: 407 = 407` ##### RangeNotSatisfiable `readonly RangeNotSatisfiable: 416 = 416` ##### RequestHeaderFieldsTooLarge `readonly RequestHeaderFieldsTooLarge: 431 = 431` ##### RequestTimeout `readonly RequestTimeout: 408 = 408` ##### ResetContent `readonly ResetContent: 205 = 205` ##### SeeOther `readonly SeeOther: 303 = 303` ##### ServiceUnavailable `readonly ServiceUnavailable: 503 = 503` ##### SwitchingProtocols `readonly SwitchingProtocols: 101 = 101` ##### SwitchProxy `readonly SwitchProxy: 306 = 306` ##### TemporaryRedirect `readonly TemporaryRedirect: 307 = 307` ##### TooEarly `readonly TooEarly: 425 = 425` ##### TooManyRequests `readonly TooManyRequests: 429 = 429` ##### Unauthorized `readonly Unauthorized: 401 = 401` ##### UnavailableForLegalReasons `readonly UnavailableForLegalReasons: 451 = 451` ##### UnprocessableEntity `readonly UnprocessableEntity: 422 = 422` ##### UnsupportedMediaType `readonly UnsupportedMediaType: 415 = 415` ##### UpgradeRequired `readonly UpgradeRequired: 426 = 426` ##### URITooLong `readonly URITooLong: 414 = 414` ##### UseProxy `readonly UseProxy: 305 = 305` ##### VariantAlsoNegotiates `readonly VariantAlsoNegotiates: 506 = 506` ## Functions ### api() #### Call Signature `function api(options, fn): HandlerFn` ##### Type Parameters ###### Params `Params` *extends* `void` \| `object` = `void` ###### Response `Response` *extends* `void` \| `object` = `void` ##### Parameters ###### options [`APIOptions`](#apioptions) ###### fn (`params`) => `Promise`\<`Response`\> ##### Returns `HandlerFn`\<`Params`, `Response`\> #### Call Signature `function api(options, fn): HandlerFn` ##### Type Parameters ###### Params `Params` *extends* `void` \| `object` = `void` ###### Response `Response` *extends* `void` \| `object` = `void` ##### Parameters ###### options [`APIOptions`](#apioptions) ###### fn (`params`) => `Response` ##### Returns `HandlerFn`\<`Params`, `Response`\> *** ### middleware() #### Call Signature `function middleware(m): Middleware` ##### Parameters ###### m [`MiddlewareFn`](#middlewarefn) ##### Returns [`Middleware`](#middleware) #### Call Signature `function middleware(options, fn): Middleware` ##### Parameters ###### options [`MiddlewareOptions`](#middlewareoptions-1) ###### fn [`MiddlewareFn`](#middlewarefn) ##### Returns [`Middleware`](#middleware) --- Source: https://encore.dev/docs/ts/runtime/auth Section: Building (Encore.ts) # encore.dev/auth ## Type Aliases ### AuthHandler `type AuthHandler = (params) => Promise & AuthHandlerBrand` #### Type Parameters ##### Params `Params` *extends* `object` ##### AuthData `AuthData` *extends* \{ `userID`: `string`; \} *** ### AuthHandlerBrand ```ts type AuthHandlerBrand = { __authHandlerBrand: unique symbol; }; ``` #### Properties ##### \_\_authHandlerBrand `readonly __authHandlerBrand: unique symbol` ## Functions ### authHandler() `function authHandler(fn): AuthHandler` #### Type Parameters ##### Params `Params` *extends* `object` ##### AuthData `AuthData` *extends* \{ `userID`: `string`; \} #### Parameters ##### fn (`params`) => `Promise`\<`AuthData` \| `null`\> #### Returns [`AuthHandler`](#authhandler)\<`Params`, `AuthData`\> --- Source: https://encore.dev/docs/ts/runtime/config Section: Building (Encore.ts) # encore.dev/config ## Interfaces ### Secret() Secret represents a single secret value that is loaded into the application. It is strongly typed for that secret, so that you can write functions which expect a specific one. You can use [AnySecret](#anysecret) to represent any secret without knowing it's name. #### Example ```ts function doFoo(s: Secret<"foo">): void { const foo = s(); } ``` #### Type Parameters ##### Name `Name` *extends* `string` `Secret(): string` Returns the current value of the secret. Encore will periodically refresh the value of the secret, so this value may change over time and could be stale for upto a couple of minutes. If you need to ensure you have the latest value, use `latest`. #### Returns `string` #### Properties ##### name `readonly name: Name` The name of the secret. ## Type Aliases ### AnySecret `type AnySecret = Secret` AnySecret is a type which can be used to represent any [Secret](#secret) without knowing its name. ## Functions ### secret() `function secret(name): Secret` secret is used to load a single [Secret](#secret) into the application. If you wish to load multiple secrets at once, see `secrets`. #### Type Parameters ##### Name `Name` *extends* `string` #### Parameters ##### name `StringLiteral`\<`Name`\> #### Returns [`Secret`](#secret)\<`Name`\> #### Example ```ts loading a single secret import {secret} from "encore.dev/config/secrets"; const foo = secret<"foo">(); ``` --- Source: https://encore.dev/docs/ts/runtime/cron Section: Building (Encore.ts) # encore.dev/cron ## Classes ### CronJob #### Constructors ##### Constructor `new CronJob(name, cfg): CronJob` ###### Parameters ###### name `string` ###### cfg [`CronJobConfig`](#cronjobconfig) ###### Returns [`CronJob`](#cronjob) #### Properties ##### cfg `readonly cfg: CronJobConfig` ##### name `readonly name: string` ## Type Aliases ### CronJobConfig ```ts type CronJobConfig = { endpoint: () => Promise; title?: string; } & | { every: DurationString; } | { schedule: string; }; ``` #### Type Declaration ##### endpoint `endpoint: () => Promise` ###### Returns `Promise`\<`unknown`\> ##### title? `optional title?: string` --- Source: https://encore.dev/docs/ts/runtime/log Section: Building (Encore.ts) # encore.dev/log ## Enumerations ### LogLevel #### Enumeration Members ##### Debug `Debug: number` ##### Error `Error: number` ##### Info `Info: number` ##### Trace `Trace: number` ##### Warn `Warn: number` ## Interfaces ### Logger #### Methods ##### debug() `debug(msg, fields?): void` Debug logs a message at the debug level. ###### Parameters ###### msg `string` ###### fields? `any` ###### Returns `void` ##### error() ###### Call Signature `error(err, fields?): void` ###### Parameters ###### err `unknown` ###### fields? `any` ###### Returns `void` ###### Call Signature ```ts error( err, msg, fields?): void; ``` ###### Parameters ###### err `unknown` ###### msg `string` ###### fields? `any` ###### Returns `void` ###### Call Signature `error(msg, fields?): void` ###### Parameters ###### msg `string` ###### fields? `any` ###### Returns `void` ##### info() `info(msg, fields?): void` Info logs a message at the info level. ###### Parameters ###### msg `string` ###### fields? `any` ###### Returns `void` ##### trace() `trace(msg, fields?): void` Trace logs a message at the trace level. ###### Parameters ###### msg `string` ###### fields? `any` ###### Returns `void` ##### warn() ###### Call Signature `warn(err, fields?): void` Warn logs a message at the warn level. ###### Parameters ###### err `unknown` ###### fields? `any` ###### Returns `void` ###### Call Signature ```ts warn( err, msg, fields?): void; ``` Warn logs a message at the warn level. ###### Parameters ###### err `unknown` ###### msg `string` ###### fields? `any` ###### Returns `void` ###### Call Signature `warn(msg, fields?): void` Warn logs a message at the warn level. ###### Parameters ###### msg `string` ###### fields? `any` ###### Returns `void` ##### with() `with(fields): Logger` Returns a new logger with the given fields added to the context. ###### Parameters ###### fields `any` ###### Returns [`Logger`](#logger) ##### withLevel() `withLevel(level): Logger` Returns a new logger with the specified level. ###### Parameters ###### level [`LogLevel`](#loglevel) ###### Returns [`Logger`](#logger) ## Type Aliases ### FieldsObject `type FieldsObject = Record` A map of fields that can be logged *** ### FieldValue ```ts type FieldValue = | string | number | boolean | null | undefined | FieldsObject | FieldValue[]; ``` A field value we support logging ## Variables ### default `const default: Logger` ## Functions ### debug() `function debug(msg, fields?): void` Debug logs a message at the debug level #### Parameters ##### msg `string` ##### fields? `any` #### Returns `void` *** ### error() #### Call Signature `function error(err, fields?): void` Error logs a message at the error level ##### Parameters ###### err `unknown` ###### fields? `any` ##### Returns `void` #### Call Signature ```ts function error( err, msg, fields?): void; ``` Error logs a message at the error level ##### Parameters ###### err `unknown` ###### msg `string` ###### fields? `any` ##### Returns `void` #### Call Signature `function error(msg, fields?): void` Error logs a message at the error level ##### Parameters ###### msg `string` ###### fields? `any` ##### Returns `void` *** ### info() `function info(msg, fields?): void` Info logs a message at the info level #### Parameters ##### msg `string` ##### fields? `any` #### Returns `void` *** ### trace() `function trace(msg, fields?): void` Trace logs a message at the trace level #### Parameters ##### msg `string` ##### fields? `any` #### Returns `void` *** ### warn() #### Call Signature `function warn(err, fields?): void` Warn logs a message at the warn level ##### Parameters ###### err `unknown` ###### fields? `any` ##### Returns `void` #### Call Signature ```ts function warn( err, msg, fields?): void; ``` Warn logs a message at the warn level ##### Parameters ###### err `unknown` ###### msg `string` ###### fields? `any` ##### Returns `void` #### Call Signature `function warn(msg, fields?): void` Warn logs a message at the warn level ##### Parameters ###### msg `string` ###### fields? `any` ##### Returns `void` --- Source: https://encore.dev/docs/ts/runtime/metrics Section: Building (Encore.ts) # encore.dev/metrics ## Classes ### Counter A Counter tracks cumulative values that only increase. Use counters for metrics like request counts, errors, etc. #### Constructors ##### Constructor `new Counter(name, cfg?): Counter` ###### Parameters ###### name `string` ###### cfg? [`MetricConfig`](#metricconfig) ###### Returns [`Counter`](#counter) #### Methods ##### increment() `increment(value?): void` Increment the counter by the given value (default 1). ###### Parameters ###### value? `number` = `1` ###### Returns `void` ##### ref() `ref(): Counter` ###### Returns [`Counter`](#counter) *** ### CounterGroup A CounterGroup tracks counters with labels. Each unique combination of label values creates a separate counter time series. #### Type Parameters ##### L `L` *extends* `Record`\ The label interface (must have string/number/boolean fields) Note: Number values in labels are converted to integers using Math.floor(). #### Constructors ##### Constructor `new CounterGroup(name, cfg?): CounterGroup` ###### Parameters ###### name `string` ###### cfg? [`MetricConfig`](#metricconfig) ###### Returns [`CounterGroup`](#countergroup)\<`L`\> #### Methods ##### ref() `ref(): CounterGroup` ###### Returns [`CounterGroup`](#countergroup)\<`L`\> ##### with() `with(labels): Counter` Get a counter for the given label values. Note: Number values in labels are converted to integers using Math.floor(). ###### Parameters ###### labels `L` ###### Returns [`Counter`](#counter) *** ### Gauge A Gauge tracks values that can go up or down. Use gauges for metrics like memory usage, active connections, temperature, etc. #### Constructors ##### Constructor `new Gauge(name, cfg?): Gauge` ###### Parameters ###### name `string` ###### cfg? [`MetricConfig`](#metricconfig) ###### Returns [`Gauge`](#gauge) #### Methods ##### ref() `ref(): Gauge` ###### Returns [`Gauge`](#gauge) ##### set() `set(value): void` Set the gauge to the given value. ###### Parameters ###### value `number` ###### Returns `void` *** ### GaugeGroup #### Type Parameters ##### L `L` *extends* `Record`\ #### Constructors ##### Constructor `new GaugeGroup(name, cfg?): GaugeGroup` ###### Parameters ###### name `string` ###### cfg? [`MetricConfig`](#metricconfig) ###### Returns [`GaugeGroup`](#gaugegroup)\<`L`\> #### Methods ##### ref() `ref(): GaugeGroup` ###### Returns [`GaugeGroup`](#gaugegroup)\<`L`\> ##### with() `with(labels): Gauge` Get a gauge for the given label values. Note: Number values in labels are converted to integers using Math.floor(). ###### Parameters ###### labels `L` ###### Returns [`Gauge`](#gauge) ## Interfaces ### MetricConfig --- Source: https://encore.dev/docs/ts/runtime/pubsub Section: Building (Encore.ts) # encore.dev/pubsub ## Classes ### Subscription #### Type Parameters ##### Msg `Msg` *extends* `object` #### Constructors ##### Constructor ```ts new Subscription( topic, name, cfg): Subscription; ``` ###### Parameters ###### topic [`Topic`](#topic)\<`Msg`\> ###### name `string` ###### cfg [`SubscriptionConfig`](#subscriptionconfig)\<`Msg`\> ###### Returns [`Subscription`](#subscription)\<`Msg`\> *** ### Topic A topic is a resource to which you can publish messages to be delivered to subscribers of that topic. #### Extends - [`TopicPerms`](#topicperms) #### Type Parameters ##### Msg `Msg` *extends* `object` #### Implements - [`Publisher`](#publisher)\<`Msg`\> #### Constructors ##### Constructor `new Topic(name, cfg): Topic` ###### Parameters ###### name `string` ###### cfg [`TopicConfig`](#topicconfig)\<`Msg`\> ###### Returns [`Topic`](#topic)\<`Msg`\> ###### Overrides `TopicPerms.constructor` #### Properties ##### cfg `readonly cfg: TopicConfig` ##### name `readonly name: string` #### Methods ##### publish() `publish(msg): Promise` ###### Parameters ###### msg `Msg` ###### Returns `Promise`\<`string`\> ###### Implementation of [`Publisher`](#publisher).[`publish`](#publish-1) ##### ref() `ref

(): P` ###### Type Parameters ###### P `P` *extends* [`TopicPerms`](#topicperms) ###### Returns `P` ## Interfaces ### Publisher #### Extends - [`TopicPerms`](#topicperms) #### Type Parameters ##### Msg `Msg` *extends* `object` #### Methods ##### publish() `abstract publish(msg): Promise` ###### Parameters ###### msg `Msg` ###### Returns `Promise`\<`string`\> *** ### RetryPolicy RetryPolicy defines how a subscription should handle retries after errors either delivering the message or processing the message. The values given to this structure are parsed at compile time, such that the correct Cloud resources can be provisioned to support the queue. As such the values given here may be clamped to the supported values by the target cloud. (i.e. min/max values brought within the supported range by the target cloud). #### Properties ##### maxBackoff? `optional maxBackoff?: DurationString` The maximum time to wait between retries. Defaults to 10 minutes. ##### maxRetries? `optional maxRetries?: number` MaxRetries is used to control deadletter queuing logic, when: n == 0: A default value of 100 retries will be used n > 0: Encore will forward a message to a dead letter queue after n retries n == pubsub.InfiniteRetries: Messages will not be forwarded to the dead letter queue by the Encore framework ##### minBackoff? `optional minBackoff?: DurationString` The minimum time to wait between retries. Defaults to 10 seconds. *** ### SubscriptionConfig SubscriptionConfig is used when creating a subscription The values given here may be clamped to the supported values by the target cloud. (i.e. ack deadline may be brought within the supported range by the target cloud pubsub implementation). #### Type Parameters ##### Msg `Msg` #### Properties ##### ackDeadline? `optional ackDeadline?: DurationString` AckDeadline is the time a consumer has to process a message before it's returned to the subscription Default is 30 seconds, however the ack deadline must be at least 1 second. ##### handler `handler: (msg) => Promise` Handler is the function which will be called to process a message sent on the topic. When this function returns an error the message will be negatively acknowledged (nacked), which will cause a redelivery attempt to be made (unless the retry policy's MaxRetries has been reached). ###### Parameters ###### msg `Msg` ###### Returns `Promise`\<`unknown`\> ##### maxConcurrency? `optional maxConcurrency?: number` MaxConcurrency is the maximum number of messages which will be processed simultaneously per instance of the service for this subscription. Note that this is per instance of the service, so if your service has scaled to 10 instances and this is set to 10, then 100 messages could be processed simultaneously. If the value is negative, then there will be no limit on the number of messages processed simultaneously. Note: This is not supported by all cloud providers; specifically on GCP when using Cloud Run instances on an unordered topic the subscription will be configured as a Push Subscription and will have an adaptive concurrency See [GCP Push Delivery Rate](https://cloud.google.com/pubsub/docs/push#push_delivery_rate). This setting also has no effect on Encore Cloud environments. If not set, it uses a reasonable default based on the cloud provider. ##### messageRetention? `optional messageRetention?: DurationString` MessageRetention is how long an undelivered message is kept on the topic before it's purged. Default is 7 days. ##### retryPolicy? `optional retryPolicy?: RetryPolicy` RetryPolicy defines how a message should be retried when the subscriber returns an error *** ### TopicConfig TopicConfig is used when creating a Topic #### Type Parameters ##### Msg `Msg` *extends* `object` #### Properties ##### deliveryGuarantee `deliveryGuarantee: DeliveryGuarantee` DeliveryGuarantee is used to configure the delivery guarantee of a Topic ##### orderingAttribute? `optional orderingAttribute?: keyof { [Key in string | number | symbol as Extract | brandedAttribute | brandedAttribute | brandedAttribute> extends never ? never : Key]: never }` OrderingAttribute is the message attribute to use as a ordering key for messages and delivery will ensure that messages with the same value will be delivered in the order they where published. If OrderingAttribute is not set, messages can be delivered in any order. It is important to note, that in the case of an error being returned by a subscription handler, the message will be retried before any subsequent messages for that ordering key are delivered. This means depending on the retry configuration, a large backlog of messages for a given ordering key may build up. When using OrderingAttribute, it is recommended to use reason about your failure modes and set the retry configuration appropriately. Once the maximum number of retries has been reached, the message will be forwarded to the dead letter queue, and the next message for that ordering key will be delivered. To create attributes on a message, use the `Attribute` type: type UserEvent = \{ user_id: Attribute<string>; action: string; } const topic = new Topic<UserEvent>("user-events", \{ deliveryGuarantee: DeliveryGuarantee.AtLeastOnce, orderingAttribute: "user_id", // Messages with the same user-id will be delivered in the order they where published }) topic.publish(ctx, \{user_id: "1", action: "login"\}) // This message will be delivered before the logout topic.publish(ctx, \{user_id: "2", action: "login"\}) // This could be delivered at any time because it has a different user id topic.publish(ctx, \{user_id: "1", action: "logout"\}) // This message will be delivered after the first message By using OrderingAttribute, the throughput will be limited depending on the cloud provider: - AWS: 300 messages per second for the topic (see [AWS SQS Quotas]). - GCP: 1MB/s for each ordering key (see [GCP PubSub Quotas]). Note: OrderingAttribute currently has no effect during local development. [AWS SQS Quotas]: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html [GCP PubSub Quotas]: https://cloud.google.com/pubsub/quotas#resource_limits *** ### TopicPerms #### Extended by - [`Topic`](#topic) - [`Publisher`](#publisher) ## Type Aliases ### Attribute `type Attribute = T | brandedAttribute` Attribute represents a field on a message that should be sent as an attribute in a PubSub message, rather than in the message body. This is useful for ordering messages, or for filtering messages on a subscription - otherwise you should not use this. To create attributes on a message, use the `Attribute` type: type Message = \{ user_id: Attribute<number>; name: string; }; const msg: Message = \{ user_id: 123, name: "John Doe", }; The union of brandedAttribute is simply used to help the TypeScript compiler understand that the type is an attribute and allow the AttributesOf type to extract the keys of said type. #### Type Parameters ##### T `T` *extends* `string` \| `number` \| `boolean` *** ### AttributesOf `type AttributesOf = keyof { [Key in keyof T as Extract extends never ? never : Key]: never }` AttributesOf is a helper type to extract all keys from an object who's type is an Attribute type. For example: type Message = \{ user_id: Attribute<number>; name: string; age: Attribute<number>; }; type MessageAttributes = AttributesOf<Message>; // "user_id" | "age" #### Type Parameters ##### T `T` *extends* `object` *** ### DeliveryGuarantee `type DeliveryGuarantee = "at-least-once" | "exactly-once"` DeliveryGuarantee is used to configure the delivery contract for a topic. ## References ### DurationString Re-exports [DurationString](https://encore.dev/ts/runtime/types.mod#durationstring) --- Source: https://encore.dev/docs/ts/runtime/service Section: Building (Encore.ts) # encore.dev/service ## Classes ### Service Defines an Encore backend service. Use this class to define a new backend service with the given name. The scope of the service is its containing directory, and all subdirectories. It must be called from files named `encore.service.ts`, to enable Encore to efficiently identify possible service definitions. #### Constructors ##### Constructor `new Service(name, cfg?): Service` ###### Parameters ###### name `string` ###### cfg? [`ServiceConfig`](#serviceconfig) ###### Returns [`Service`](#service) #### Properties ##### cfg `readonly cfg: ServiceConfig` ##### name `readonly name: string` ## Interfaces ### ServiceConfig #### Properties ##### middlewares? `optional middlewares?: Middleware[]` --- Source: https://encore.dev/docs/ts/runtime/storage-cache Section: Building (Encore.ts) # encore.dev/storage/cache ## Classes ### CacheCluster CacheCluster represents a Redis cache cluster. Create a new cluster using `new CacheCluster(name)`. Reference an existing cluster using `CacheCluster.named(name)`. #### Example ```ts import { CacheCluster } from "encore.dev/storage/cache"; const myCache = new CacheCluster("my-cache", { evictionPolicy: "allkeys-lru", }); ``` #### Constructors ##### Constructor `new CacheCluster(name, cfg?): CacheCluster` Creates a new cache cluster with the given name and configuration. ###### Parameters ###### name `string` The unique name for this cache cluster ###### cfg? [`CacheClusterConfig`](#cacheclusterconfig) Optional configuration for the cluster ###### Returns [`CacheCluster`](#cachecluster) #### Methods ##### named() `static named(name): CacheCluster` Reference an existing cache cluster by name. To create a new cache cluster, use `new CacheCluster(...)` instead. ###### Type Parameters ###### Name `Name` *extends* `string` ###### Parameters ###### name `StringLiteral`\<`Name`\> ###### Returns [`CacheCluster`](#cachecluster) *** ### CacheError CacheError is the base class for all cache-related errors. #### Extends - `Error` #### Extended by - [`CacheMiss`](#cachemiss) - [`CacheKeyExists`](#cachekeyexists) #### Constructors ##### Constructor `new CacheError(msg): CacheError` ###### Parameters ###### msg `string` ###### Returns [`CacheError`](#cacheerror) ###### Overrides `Error.constructor` *** ### CacheKeyExists CacheKeyExists is thrown when attempting to set a key that already exists using setIfNotExists. #### Extends - [`CacheError`](#cacheerror) #### Constructors ##### Constructor `new CacheKeyExists(key): CacheKeyExists` ###### Parameters ###### key `string` ###### Returns [`CacheKeyExists`](#cachekeyexists) ###### Overrides [`CacheError`](#cacheerror).[`constructor`](#constructor-1) *** ### CacheMiss CacheMiss is thrown when a cache key is not found. #### Extends - [`CacheError`](#cacheerror) #### Constructors ##### Constructor `new CacheMiss(key): CacheMiss` ###### Parameters ###### key `string` ###### Returns [`CacheMiss`](#cachemiss) ###### Overrides [`CacheError`](#cacheerror).[`constructor`](#constructor-1) *** ### FloatKeyspace FloatKeyspace stores 64-bit floating point values. #### Example ```ts const scores = new FloatKeyspace(cluster, { keyPattern: "score/:playerId", }); await scores.set("player1", 100.5); const newScore = await scores.increment("player1", 10.25); ``` #### Extends - `BasicKeyspace`\<`K`, `number`\> #### Type Parameters ##### K `K` #### Constructors ##### Constructor `new FloatKeyspace(cluster, config): FloatKeyspace` ###### Parameters ###### cluster [`CacheCluster`](#cachecluster) ###### config [`KeyspaceConfig`](#keyspaceconfig)\<`K`\> ###### Returns [`FloatKeyspace`](#floatkeyspace)\<`K`\> ###### Overrides `BasicKeyspace.constructor` #### Properties ##### cluster `protected readonly cluster: CacheCluster` ###### Inherited from `BasicKeyspace.cluster` ##### config `protected readonly config: KeyspaceConfig` ###### Inherited from `BasicKeyspace.config` ##### keyMapper `protected readonly keyMapper: (key) => string` ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `BasicKeyspace.keyMapper` #### Methods ##### decrement() ```ts decrement( key, delta?, options?): Promise; ``` Decrements the number stored at key by `delta`. If the key does not exist it is first created with a value of 0 before decrementing. Negative values can be used to increase the value, but typically you want to use [increment](#increment) for that. ###### Parameters ###### key `K` The cache key. ###### delta? `number` = `1` The amount to decrement by (default 1). ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new value after decrementing. ###### See https://redis.io/commands/incrbyfloat/ ##### delete() `delete(...keys): Promise` Deletes the specified keys. If a key does not exist it is ignored. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`number`\> The number of keys that were deleted. ###### See https://redis.io/commands/del/ ###### Inherited from `BasicKeyspace.delete` ##### deserialize() `protected deserialize(data): number` Deserializes a Buffer from storage to a value. ###### Parameters ###### data `Buffer` ###### Returns `number` ###### Overrides `BasicKeyspace.deserialize` ##### get() `get(key): Promise` Gets the value stored at key. If the key does not exist, it returns `undefined`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number` \| `undefined`\> The value, or `undefined` if the key does not exist. ###### See https://redis.io/commands/get/ ###### Inherited from `BasicKeyspace.get` ##### getAndDelete() `getAndDelete(key): Promise` Deletes the key and returns the previously stored value. If the key does not already exist, it returns `undefined`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number` \| `undefined`\> The previous value, or `undefined` if the key did not exist. ###### See https://redis.io/commands/getdel/ ###### Inherited from `BasicKeyspace.getAndDelete` ##### getAndSet() ```ts getAndSet( key, value, options?): Promise; ``` Updates the value of key to val and returns the previously stored value. If the key does not already exist, it sets it and returns `undefined`. ###### Parameters ###### key `K` ###### value `number` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number` \| `undefined`\> The previous value, or `undefined` if the key did not exist. ###### See https://redis.io/commands/getset/ ###### Inherited from `BasicKeyspace.getAndSet` ##### increment() ```ts increment( key, delta?, options?): Promise; ``` Increments the number stored at key by `delta`. If the key does not exist it is first created with a value of 0 before incrementing. Negative values can be used to decrease the value, but typically you want to use [decrement](#decrement) for that. ###### Parameters ###### key `K` The cache key. ###### delta? `number` = `1` The amount to increment by (default 1). ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new value after incrementing. ###### See https://redis.io/commands/incrbyfloat/ ##### mapKey() `protected mapKey(key): string` Maps a key to its Redis key string. ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `BasicKeyspace.mapKey` ##### multiGet() `multiGet(...keys): Promise<(number | undefined)[]>` Gets the values stored at multiple keys. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<(`number` \| `undefined`)[]\> An array of values in the same order as the provided keys. Each element is the value or `undefined` if the key was not found. ###### See https://redis.io/commands/mget/ ###### Inherited from `BasicKeyspace.multiGet` ##### multiSet() `multiSet(entries, options?): Promise` Updates the values stored at multiple keys. The keyspace's expiry (or the expiry provided in `options`) is applied to all keys. ###### Parameters ###### entries \[`K`, `number`\][] The key-value pairs to set. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/mset/ ###### Inherited from `BasicKeyspace.multiSet` ##### replace() ```ts replace( key, value, options?): Promise; ``` Replaces the existing value stored at key to val. ###### Parameters ###### key `K` ###### value `number` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the key does not already exist. ###### See https://redis.io/commands/set/ ###### Inherited from `BasicKeyspace.replace` ##### resolveTtl() `protected resolveTtl(options?): number | undefined` Resolves the TTL for a write operation. Returns i64 sentinel for NAPI: undefined=no config, -1=KeepTTL, -2=Persist/NeverExpire, >=0=ms ###### Parameters ###### options? [`WriteOptions`](#writeoptions) ###### Returns `number` \| `undefined` ###### Inherited from `BasicKeyspace.resolveTtl` ##### serialize() `protected serialize(value): Buffer` Serializes a value to a Buffer for storage. ###### Parameters ###### value `number` ###### Returns `Buffer` ###### Overrides `BasicKeyspace.serialize` ##### set() ```ts set( key, value, options?): Promise; ``` Updates the value stored at key to val. ###### Parameters ###### key `K` ###### value `number` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/set/ ###### Inherited from `BasicKeyspace.set` ##### setIfNotExists() ```ts setIfNotExists( key, value, options?): Promise; ``` Sets the value stored at key to val, but only if the key does not exist beforehand. ###### Parameters ###### key `K` ###### value `number` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the key already exists. ###### See https://redis.io/commands/setnx/ ###### Inherited from `BasicKeyspace.setIfNotExists` ##### with() `with(options): this` Returns a shallow clone of this keyspace with the specified write options applied. This allows setting expiry for a chain of operations. ###### Parameters ###### options [`WriteOptions`](#writeoptions) ###### Returns `this` ###### Example `await myKeyspace.with({ expiry: expireIn(5000) }).set(key, value)` ###### Inherited from `BasicKeyspace.with` *** ### IntKeyspace IntKeyspace stores 64-bit integer values. Values are floored to integers using `Math.floor`. For fractional values, use [FloatKeyspace](#floatkeyspace) instead. #### Example ```ts const counters = new IntKeyspace(cluster, { keyPattern: "counter/:name", }); await counters.set("page-views", 0); const newCount = await counters.increment("page-views", 1); ``` #### Extends - `BasicKeyspace`\<`K`, `number`\> #### Type Parameters ##### K `K` #### Constructors ##### Constructor `new IntKeyspace(cluster, config): IntKeyspace` ###### Parameters ###### cluster [`CacheCluster`](#cachecluster) ###### config [`KeyspaceConfig`](#keyspaceconfig)\<`K`\> ###### Returns [`IntKeyspace`](#intkeyspace)\<`K`\> ###### Overrides `BasicKeyspace.constructor` #### Properties ##### cluster `protected readonly cluster: CacheCluster` ###### Inherited from `BasicKeyspace.cluster` ##### config `protected readonly config: KeyspaceConfig` ###### Inherited from `BasicKeyspace.config` ##### keyMapper `protected readonly keyMapper: (key) => string` ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `BasicKeyspace.keyMapper` #### Methods ##### decrement() ```ts decrement( key, delta?, options?): Promise; ``` Decrements the number stored at key by `delta`. If the key does not exist it is first created with a value of 0 before decrementing. Negative values can be used to increase the value, but typically you want to use [increment](#increment-1) for that. ###### Parameters ###### key `K` The cache key. ###### delta? `number` = `1` The amount to decrement by (default 1). ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new value after decrementing. ###### See https://redis.io/commands/decrby/ ##### delete() `delete(...keys): Promise` Deletes the specified keys. If a key does not exist it is ignored. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`number`\> The number of keys that were deleted. ###### See https://redis.io/commands/del/ ###### Inherited from `BasicKeyspace.delete` ##### deserialize() `protected deserialize(data): number` Deserializes a Buffer from storage to a value. ###### Parameters ###### data `Buffer` ###### Returns `number` ###### Overrides `BasicKeyspace.deserialize` ##### get() `get(key): Promise` Gets the value stored at key. If the key does not exist, it returns `undefined`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number` \| `undefined`\> The value, or `undefined` if the key does not exist. ###### See https://redis.io/commands/get/ ###### Inherited from `BasicKeyspace.get` ##### getAndDelete() `getAndDelete(key): Promise` Deletes the key and returns the previously stored value. If the key does not already exist, it returns `undefined`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number` \| `undefined`\> The previous value, or `undefined` if the key did not exist. ###### See https://redis.io/commands/getdel/ ###### Inherited from `BasicKeyspace.getAndDelete` ##### getAndSet() ```ts getAndSet( key, value, options?): Promise; ``` Updates the value of key to val and returns the previously stored value. If the key does not already exist, it sets it and returns `undefined`. ###### Parameters ###### key `K` ###### value `number` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number` \| `undefined`\> The previous value, or `undefined` if the key did not exist. ###### See https://redis.io/commands/getset/ ###### Inherited from `BasicKeyspace.getAndSet` ##### increment() ```ts increment( key, delta?, options?): Promise; ``` Increments the number stored at key by `delta`. If the key does not exist it is first created with a value of 0 before incrementing. Negative values can be used to decrease the value, but typically you want to use [decrement](#decrement-1) for that. ###### Parameters ###### key `K` The cache key. ###### delta? `number` = `1` The amount to increment by (default 1). ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new value after incrementing. ###### See https://redis.io/commands/incrby/ ##### mapKey() `protected mapKey(key): string` Maps a key to its Redis key string. ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `BasicKeyspace.mapKey` ##### multiGet() `multiGet(...keys): Promise<(number | undefined)[]>` Gets the values stored at multiple keys. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<(`number` \| `undefined`)[]\> An array of values in the same order as the provided keys. Each element is the value or `undefined` if the key was not found. ###### See https://redis.io/commands/mget/ ###### Inherited from `BasicKeyspace.multiGet` ##### multiSet() `multiSet(entries, options?): Promise` Updates the values stored at multiple keys. The keyspace's expiry (or the expiry provided in `options`) is applied to all keys. ###### Parameters ###### entries \[`K`, `number`\][] The key-value pairs to set. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/mset/ ###### Inherited from `BasicKeyspace.multiSet` ##### replace() ```ts replace( key, value, options?): Promise; ``` Replaces the existing value stored at key to val. ###### Parameters ###### key `K` ###### value `number` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the key does not already exist. ###### See https://redis.io/commands/set/ ###### Inherited from `BasicKeyspace.replace` ##### resolveTtl() `protected resolveTtl(options?): number | undefined` Resolves the TTL for a write operation. Returns i64 sentinel for NAPI: undefined=no config, -1=KeepTTL, -2=Persist/NeverExpire, >=0=ms ###### Parameters ###### options? [`WriteOptions`](#writeoptions) ###### Returns `number` \| `undefined` ###### Inherited from `BasicKeyspace.resolveTtl` ##### serialize() `protected serialize(value): Buffer` Serializes a value to a Buffer for storage. ###### Parameters ###### value `number` ###### Returns `Buffer` ###### Overrides `BasicKeyspace.serialize` ##### set() ```ts set( key, value, options?): Promise; ``` Updates the value stored at key to val. ###### Parameters ###### key `K` ###### value `number` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/set/ ###### Inherited from `BasicKeyspace.set` ##### setIfNotExists() ```ts setIfNotExists( key, value, options?): Promise; ``` Sets the value stored at key to val, but only if the key does not exist beforehand. ###### Parameters ###### key `K` ###### value `number` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the key already exists. ###### See https://redis.io/commands/setnx/ ###### Inherited from `BasicKeyspace.setIfNotExists` ##### with() `with(options): this` Returns a shallow clone of this keyspace with the specified write options applied. This allows setting expiry for a chain of operations. ###### Parameters ###### options [`WriteOptions`](#writeoptions) ###### Returns `this` ###### Example `await myKeyspace.with({ expiry: expireIn(5000) }).set(key, value)` ###### Inherited from `BasicKeyspace.with` *** ### NumberListKeyspace NumberListKeyspace stores lists of numeric values. #### Example ```ts const scores = new NumberListKeyspace(cluster, { keyPattern: "scores/:gameId", }); await scores.pushRight("game1", 100, 200, 300); const allScores = await scores.items("game1"); ``` #### Extends - `ListKeyspace`\<`K`, `number`\> #### Type Parameters ##### K `K` #### Constructors ##### Constructor `new NumberListKeyspace(cluster, config): NumberListKeyspace` ###### Parameters ###### cluster [`CacheCluster`](#cachecluster) ###### config [`KeyspaceConfig`](#keyspaceconfig)\<`K`\> ###### Returns [`NumberListKeyspace`](#numberlistkeyspace)\<`K`\> ###### Overrides `ListKeyspace.constructor` #### Properties ##### cluster `protected readonly cluster: CacheCluster` ###### Inherited from `ListKeyspace.cluster` ##### config `protected readonly config: KeyspaceConfig` ###### Inherited from `ListKeyspace.config` ##### keyMapper `protected readonly keyMapper: (key) => string` ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `ListKeyspace.keyMapper` #### Methods ##### delete() `delete(...keys): Promise` Deletes the specified keys. If a key does not exist it is ignored. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`number`\> The number of keys that were deleted. ###### See https://redis.io/commands/del/ ###### Inherited from `ListKeyspace.delete` ##### deserializeItem() `protected deserializeItem(data): number` ###### Parameters ###### data `Buffer` ###### Returns `number` ###### Overrides `ListKeyspace.deserializeItem` ##### get() `get(key, index): Promise` Returns the value of the list element at the given index. Negative indices can be used to indicate offsets from the end of the list, where -1 is the last element of the list, -2 the penultimate element, and so on. ###### Parameters ###### key `K` The cache key. ###### index `number` Zero-based index of the element to retrieve. ###### Returns `Promise`\<`number` \| `undefined`\> The value at the index, or `undefined` if out of range or the key does not exist. ###### See https://redis.io/commands/lindex/ ###### Inherited from `ListKeyspace.get` ##### getRange() ```ts getRange( key, start, stop): Promise; ``` Returns the elements in the list stored at key between `start` and `stop` (inclusive). Both are zero-based indices. Negative indices can be used to indicate offsets from the end of the list, where -1 is the last element of the list, -2 the penultimate element, and so on. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` The cache key. ###### start `number` Start index (inclusive). ###### stop `number` Stop index (inclusive). ###### Returns `Promise`\<`number`[]\> The elements in the specified range. ###### See https://redis.io/commands/lrange/ ###### Inherited from `ListKeyspace.getRange` ##### insertAfter() ```ts insertAfter( key, pivot, value, options?): Promise; ``` Inserts `value` into the list stored at key, at the position just after `pivot`. If the list does not contain `pivot`, the value is not inserted and -1 is returned. ###### Parameters ###### key `K` The cache key. ###### pivot `number` The existing element to insert after. ###### value `number` The value to insert. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new list length, or -1 if `pivot` was not found. ###### See https://redis.io/commands/linsert/ ###### Inherited from `ListKeyspace.insertAfter` ##### insertBefore() ```ts insertBefore( key, pivot, value, options?): Promise; ``` Inserts `value` into the list stored at key, at the position just before `pivot`. If the list does not contain `pivot`, the value is not inserted and -1 is returned. ###### Parameters ###### key `K` The cache key. ###### pivot `number` The existing element to insert before. ###### value `number` The value to insert. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new list length, or -1 if `pivot` was not found. ###### See https://redis.io/commands/linsert/ ###### Inherited from `ListKeyspace.insertBefore` ##### items() `items(key): Promise` Returns all the elements in the list stored at key. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number`[]\> All elements in the list. ###### See https://redis.io/commands/lrange/ ###### Inherited from `ListKeyspace.items` ##### len() `len(key): Promise` Returns the length of the list stored at key. Non-existing keys are considered as empty lists. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number`\> The list length. ###### See https://redis.io/commands/llen/ ###### Inherited from `ListKeyspace.len` ##### mapKey() `protected mapKey(key): string` Maps a key to its Redis key string. ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `ListKeyspace.mapKey` ##### move() ```ts move( src, dst, srcPos, dstPos, options?): Promise; ``` Atomically moves an element from the list stored at `src` to the list stored at `dst`. The value moved can be either the head (`srcPos === "left"`) or tail (`srcPos === "right"`) of the list at `src`. Similarly, the value can be placed either at the head (`dstPos === "left"`) or tail (`dstPos === "right"`) of the list at `dst`. If `src` and `dst` are the same list, the value is atomically rotated from one end to the other when `srcPos !== dstPos`, or if `srcPos === dstPos` nothing happens. ###### Parameters ###### src `K` Source list key. ###### dst `K` Destination list key. ###### srcPos [`ListPosition`](#listposition) Position to pop from in the source list. ###### dstPos [`ListPosition`](#listposition) Position to push to in the destination list. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number` \| `undefined`\> The moved element, or `undefined` if the source list does not exist. ###### See https://redis.io/commands/lmove/ ###### Inherited from `ListKeyspace.move` ##### popLeft() `popLeft(key, options?): Promise` Pops a single element off the head of the list stored at key. ###### Parameters ###### key `K` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number` \| `undefined`\> The popped value, or `undefined` if the key does not exist. ###### See https://redis.io/commands/lpop/ ###### Inherited from `ListKeyspace.popLeft` ##### popRight() `popRight(key, options?): Promise` Pops a single element off the tail of the list stored at key. ###### Parameters ###### key `K` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number` \| `undefined`\> The popped value, or `undefined` if the key does not exist. ###### See https://redis.io/commands/rpop/ ###### Inherited from `ListKeyspace.popRight` ##### pushLeft() `pushLeft(key, ...values): Promise` Pushes one or more values at the head of the list stored at key. If the key does not already exist, it is first created as an empty list. If multiple values are given, they are inserted one after another, starting with the leftmost value. For instance, `pushLeft(key, "a", "b", "c")` will result in a list containing "c" as its first element, "b" as its second, and "a" as its third. ###### Parameters ###### key `K` ###### values ...`number`[] ###### Returns `Promise`\<`number`\> The length of the list after the operation. ###### See https://redis.io/commands/lpush/ ###### Inherited from `ListKeyspace.pushLeft` ##### pushRight() `pushRight(key, ...values): Promise` Pushes one or more values at the tail of the list stored at key. If the key does not already exist, it is first created as an empty list. If multiple values are given, they are inserted one after another, starting with the leftmost value. For instance, `pushRight(key, "a", "b", "c")` will result in a list containing "a" as its first element, "b" as its second, and "c" as its third. ###### Parameters ###### key `K` ###### values ...`number`[] ###### Returns `Promise`\<`number`\> The length of the list after the operation. ###### See https://redis.io/commands/rpush/ ###### Inherited from `ListKeyspace.pushRight` ##### removeAll() ```ts removeAll( key, value, options?): Promise; ``` Removes all occurrences of `value` in the list stored at key. If the list does not contain `value`, or the list does not exist, returns 0. ###### Parameters ###### key `K` The cache key. ###### value `number` The value to remove. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The number of elements removed. ###### See https://redis.io/commands/lrem/ ###### Inherited from `ListKeyspace.removeAll` ##### removeFirst() ```ts removeFirst( key, count, value, options?): Promise; ``` Removes the first `count` occurrences of `value` in the list stored at key, scanning from head to tail. If the list does not contain `value`, or the list does not exist, returns 0. ###### Parameters ###### key `K` The cache key. ###### count `number` Maximum number of occurrences to remove. ###### value `number` The value to remove. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The number of elements removed. ###### See https://redis.io/commands/lrem/ ###### Inherited from `ListKeyspace.removeFirst` ##### removeLast() ```ts removeLast( key, count, value, options?): Promise; ``` Removes the last `count` occurrences of `value` in the list stored at key, scanning from tail to head. If the list does not contain `value`, or the list does not exist, returns 0. ###### Parameters ###### key `K` The cache key. ###### count `number` Maximum number of occurrences to remove. ###### value `number` The value to remove. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The number of elements removed. ###### See https://redis.io/commands/lrem/ ###### Inherited from `ListKeyspace.removeLast` ##### resolveTtl() `protected resolveTtl(options?): number | undefined` Resolves the TTL for a write operation. Returns i64 sentinel for NAPI: undefined=no config, -1=KeepTTL, -2=Persist/NeverExpire, >=0=ms ###### Parameters ###### options? [`WriteOptions`](#writeoptions) ###### Returns `number` \| `undefined` ###### Inherited from `ListKeyspace.resolveTtl` ##### serializeItem() `protected serializeItem(value): Buffer` ###### Parameters ###### value `number` ###### Returns `Buffer` ###### Overrides `ListKeyspace.serializeItem` ##### set() ```ts set( key, index, value, options?): Promise; ``` Updates the list element at the given index. Negative indices can be used to indicate offsets from the end of the list, where -1 is the last element of the list, -2 the penultimate element, and so on. ###### Parameters ###### key `K` The cache key. ###### index `number` Zero-based index of the element to update. ###### value `number` The new value. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the index is out of range. ###### See https://redis.io/commands/lset/ ###### Inherited from `ListKeyspace.set` ##### trim() ```ts trim( key, start, stop, options?): Promise; ``` Trims the list stored at key to only contain the elements between the indices `start` and `stop` (inclusive). Both are zero-based indices. Negative indices can be used to indicate offsets from the end of the list, where -1 is the last element of the list, -2 the penultimate element, and so on. Out of range indices are valid and are treated as if they specify the start or end of the list, respectively. If `start` > `stop` the end result is an empty list. ###### Parameters ###### key `K` The cache key. ###### start `number` Start index (inclusive). ###### stop `number` Stop index (inclusive). ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/ltrim/ ###### Inherited from `ListKeyspace.trim` ##### with() `with(options): this` Returns a shallow clone of this keyspace with the specified write options applied. This allows setting expiry for a chain of operations. ###### Parameters ###### options [`WriteOptions`](#writeoptions) ###### Returns `this` ###### Example `await myKeyspace.with({ expiry: expireIn(5000) }).set(key, value)` ###### Inherited from `ListKeyspace.with` *** ### NumberSetKeyspace NumberSetKeyspace stores sets of unique numeric values. #### Example ```ts const scores = new NumberSetKeyspace(cluster, { keyPattern: "unique-scores/:gameId", }); await scores.add("game1", 100, 200, 300); const hasScore = await scores.contains("game1", 100); ``` #### Extends - `SetKeyspace`\<`K`, `number`\> #### Type Parameters ##### K `K` #### Constructors ##### Constructor `new NumberSetKeyspace(cluster, config): NumberSetKeyspace` ###### Parameters ###### cluster [`CacheCluster`](#cachecluster) ###### config [`KeyspaceConfig`](#keyspaceconfig)\<`K`\> ###### Returns [`NumberSetKeyspace`](#numbersetkeyspace)\<`K`\> ###### Overrides `SetKeyspace.constructor` #### Properties ##### cluster `protected readonly cluster: CacheCluster` ###### Inherited from `SetKeyspace.cluster` ##### config `protected readonly config: KeyspaceConfig` ###### Inherited from `SetKeyspace.config` ##### keyMapper `protected readonly keyMapper: (key) => string` ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `SetKeyspace.keyMapper` #### Methods ##### add() `add(key, ...members): Promise` Adds one or more values to the set stored at key. If the key does not already exist, it is first created as an empty set. ###### Parameters ###### key `K` ###### members ...`number`[] ###### Returns `Promise`\<`number`\> The number of values that were added to the set, not including values already present beforehand. ###### See https://redis.io/commands/sadd/ ###### Inherited from `SetKeyspace.add` ##### contains() `contains(key, member): Promise` Reports whether the set stored at key contains the given value. If the key does not exist it returns `false`. ###### Parameters ###### key `K` ###### member `number` ###### Returns `Promise`\<`boolean`\> `true` if the member exists in the set, `false` otherwise. ###### See https://redis.io/commands/sismember/ ###### Inherited from `SetKeyspace.contains` ##### delete() `delete(...keys): Promise` Deletes the specified keys. If a key does not exist it is ignored. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`number`\> The number of keys that were deleted. ###### See https://redis.io/commands/del/ ###### Inherited from `SetKeyspace.delete` ##### deserializeItem() `protected deserializeItem(data): number` ###### Parameters ###### data `Buffer` ###### Returns `number` ###### Overrides `SetKeyspace.deserializeItem` ##### diff() `diff(...keys): Promise` Computes the set difference between the first set and all the consecutive sets. Set difference means the values present in the first set that are not present in any of the other sets. Keys that do not exist are considered as empty sets. ###### Parameters ###### keys ...`K`[] Keys of sets to compute difference for. At least one must be provided. ###### Returns `Promise`\<`number`[]\> Members in the first set but not in any of the other sets. ###### Throws If no keys are provided. ###### See https://redis.io/commands/sdiff/ ###### Inherited from `SetKeyspace.diff` ##### diffSet() `diffSet(...keys): Promise>` Identical to [diff](#diff) except it returns the values as a `Set`. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`Set`\<`number`\>\> ###### See https://redis.io/commands/sdiff/ ###### Inherited from `SetKeyspace.diffSet` ##### diffStore() `diffStore(destination, ...keys): Promise` Computes the set difference between keys (like [diff](#diff)) and stores the result in `destination`. ###### Parameters ###### destination `K` Key to store the result. ###### keys ...`K`[] Keys of sets to compute difference for. ###### Returns `Promise`\<`number`\> The size of the resulting set. ###### See https://redis.io/commands/sdiffstore/ ###### Inherited from `SetKeyspace.diffStore` ##### intersect() `intersect(...keys): Promise` Computes the set intersection between the sets stored at the given keys. Set intersection means the values common to all the provided sets. Keys that do not exist are considered to be empty sets. As a result, if any key is missing the final result is the empty set. ###### Parameters ###### keys ...`K`[] Keys of sets to compute intersection for. At least one must be provided. ###### Returns `Promise`\<`number`[]\> Members common to all sets. ###### Throws If no keys are provided. ###### See https://redis.io/commands/sinter/ ###### Inherited from `SetKeyspace.intersect` ##### intersectSet() `intersectSet(...keys): Promise>` Identical to [intersect](#intersect) except it returns the values as a `Set`. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`Set`\<`number`\>\> ###### See https://redis.io/commands/sinter/ ###### Inherited from `SetKeyspace.intersectSet` ##### intersectStore() `intersectStore(destination, ...keys): Promise` Computes the set intersection between keys (like [intersect](#intersect)) and stores the result in `destination`. ###### Parameters ###### destination `K` Key to store the result. ###### keys ...`K`[] Keys of sets to compute intersection for. ###### Returns `Promise`\<`number`\> The size of the resulting set. ###### See https://redis.io/commands/sinterstore/ ###### Inherited from `SetKeyspace.intersectStore` ##### items() `items(key): Promise` Returns the elements in the set stored at key. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number`[]\> All members of the set. ###### See https://redis.io/commands/smembers/ ###### Inherited from `SetKeyspace.items` ##### itemsSet() `itemsSet(key): Promise>` Identical to [items](#items-1) except it returns the values as a `Set`. If the key does not exist it returns an empty `Set`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`Set`\<`number`\>\> All members of the set as a `Set`. ###### See https://redis.io/commands/smembers/ ###### Inherited from `SetKeyspace.itemsSet` ##### len() `len(key): Promise` Returns the number of elements in the set stored at key. If the key does not exist it returns 0. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number`\> The set cardinality. ###### See https://redis.io/commands/scard/ ###### Inherited from `SetKeyspace.len` ##### mapKey() `protected mapKey(key): string` Maps a key to its Redis key string. ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `SetKeyspace.mapKey` ##### move() ```ts move( src, dst, member, options?): Promise; ``` Atomically moves the given member from the set stored at `src` to the set stored at `dst`. If the element already exists in `dst` it is still removed from `src`. ###### Parameters ###### src `K` Source set key. ###### dst `K` Destination set key. ###### member `number` The member to move. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`boolean`\> `true` if the member was moved, `false` if not found in `src`. ###### See https://redis.io/commands/smove/ ###### Inherited from `SetKeyspace.move` ##### pop() ```ts pop( key, count, options?): Promise; ``` Removes up to `count` random elements (bounded by the set's size) from the set stored at key and returns them. If the set is empty it returns an empty array. ###### Parameters ###### key `K` The cache key. ###### count `number` Number of members to pop. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`[]\> The removed members (may be fewer than `count` if the set is small). ###### See https://redis.io/commands/spop/ ###### Inherited from `SetKeyspace.pop` ##### popOne() `popOne(key, options?): Promise` Removes a random element from the set stored at key and returns it. ###### Parameters ###### key `K` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number` \| `undefined`\> The removed member, or `undefined` if the set is empty. ###### See https://redis.io/commands/spop/ ###### Inherited from `SetKeyspace.popOne` ##### remove() `remove(key, ...members): Promise` Removes one or more values from the set stored at key. Values not present in the set are ignored. If the key does not already exist, it is a no-op. ###### Parameters ###### key `K` ###### members ...`number`[] ###### Returns `Promise`\<`number`\> The number of values that were removed from the set. ###### See https://redis.io/commands/srem/ ###### Inherited from `SetKeyspace.remove` ##### resolveTtl() `protected resolveTtl(options?): number | undefined` Resolves the TTL for a write operation. Returns i64 sentinel for NAPI: undefined=no config, -1=KeepTTL, -2=Persist/NeverExpire, >=0=ms ###### Parameters ###### options? [`WriteOptions`](#writeoptions) ###### Returns `number` \| `undefined` ###### Inherited from `SetKeyspace.resolveTtl` ##### sample() `sample(key, count): Promise` Returns up to `count` distinct random elements from the set stored at key. The same element is never returned multiple times. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` The cache key. ###### count `number` Number of distinct members to return. ###### Returns `Promise`\<`number`[]\> Random members (may be fewer than `count` if the set is small). ###### See https://redis.io/commands/srandmember/ ###### Inherited from `SetKeyspace.sample` ##### sampleOne() `sampleOne(key): Promise` Returns a random member from the set stored at key without removing it. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number` \| `undefined`\> A random member, or `undefined` if the key does not exist. ###### See https://redis.io/commands/srandmember/ ###### Inherited from `SetKeyspace.sampleOne` ##### sampleWithReplacement() `sampleWithReplacement(key, count): Promise` Returns `count` random elements from the set stored at key. The same element may be returned multiple times. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` The cache key. ###### count `number` Number of members to return (may include duplicates). ###### Returns `Promise`\<`number`[]\> Random members, possibly with duplicates. ###### See https://redis.io/commands/srandmember/ ###### Inherited from `SetKeyspace.sampleWithReplacement` ##### serializeItem() `protected serializeItem(value): Buffer` ###### Parameters ###### value `number` ###### Returns `Buffer` ###### Overrides `SetKeyspace.serializeItem` ##### union() `union(...keys): Promise` Computes the set union between the sets stored at the given keys. Set union means the values present in at least one of the provided sets. Keys that do not exist are considered to be empty sets. ###### Parameters ###### keys ...`K`[] Keys of sets to compute union for. At least one must be provided. ###### Returns `Promise`\<`number`[]\> Members in any of the provided sets. ###### Throws If no keys are provided. ###### See https://redis.io/commands/sunion/ ###### Inherited from `SetKeyspace.union` ##### unionSet() `unionSet(...keys): Promise>` Identical to [union](#union) except it returns the values as a `Set`. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`Set`\<`number`\>\> ###### See https://redis.io/commands/sunion/ ###### Inherited from `SetKeyspace.unionSet` ##### unionStore() `unionStore(destination, ...keys): Promise` Computes the set union between sets (like [union](#union)) and stores the result in `destination`. ###### Parameters ###### destination `K` Key to store the result. ###### keys ...`K`[] Keys of sets to compute union for. ###### Returns `Promise`\<`number`\> The size of the resulting set. ###### See https://redis.io/commands/sunionstore/ ###### Inherited from `SetKeyspace.unionStore` ##### with() `with(options): this` Returns a shallow clone of this keyspace with the specified write options applied. This allows setting expiry for a chain of operations. ###### Parameters ###### options [`WriteOptions`](#writeoptions) ###### Returns `this` ###### Example `await myKeyspace.with({ expiry: expireIn(5000) }).set(key, value)` ###### Inherited from `SetKeyspace.with` *** ### StringKeyspace StringKeyspace stores string values. #### Example ```ts const tokens = new StringKeyspace(cluster, { keyPattern: "token/:id", defaultExpiry: ExpireIn(3600000), // 1 hour }); await tokens.set("abc123", "user-token-value"); const token = await tokens.get("abc123"); ``` #### Extends - `BasicKeyspace`\<`K`, `string`\> #### Type Parameters ##### K `K` #### Constructors ##### Constructor `new StringKeyspace(cluster, config): StringKeyspace` ###### Parameters ###### cluster [`CacheCluster`](#cachecluster) ###### config [`KeyspaceConfig`](#keyspaceconfig)\<`K`\> ###### Returns [`StringKeyspace`](#stringkeyspace)\<`K`\> ###### Overrides `BasicKeyspace.constructor` #### Properties ##### cluster `protected readonly cluster: CacheCluster` ###### Inherited from [`IntKeyspace`](#intkeyspace).[`cluster`](#cluster-1) ##### config `protected readonly config: KeyspaceConfig` ###### Inherited from `BasicKeyspace.config` ##### keyMapper `protected readonly keyMapper: (key) => string` ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `BasicKeyspace.keyMapper` #### Methods ##### append() ```ts append( key, value, options?): Promise; ``` Appends a string to the value stored at key. If the key does not exist it is first created and set as the empty string, causing append to behave like set. ###### Parameters ###### key `K` ###### value `string` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new string length. ###### See https://redis.io/commands/append/ ##### delete() `delete(...keys): Promise` Deletes the specified keys. If a key does not exist it is ignored. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`number`\> The number of keys that were deleted. ###### See https://redis.io/commands/del/ ###### Inherited from `BasicKeyspace.delete` ##### deserialize() `protected deserialize(data): string` Deserializes a Buffer from storage to a value. ###### Parameters ###### data `Buffer` ###### Returns `string` ###### Overrides `BasicKeyspace.deserialize` ##### get() `get(key): Promise` Gets the value stored at key. If the key does not exist, it returns `undefined`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`string` \| `undefined`\> The value, or `undefined` if the key does not exist. ###### See https://redis.io/commands/get/ ###### Inherited from `BasicKeyspace.get` ##### getAndDelete() `getAndDelete(key): Promise` Deletes the key and returns the previously stored value. If the key does not already exist, it returns `undefined`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`string` \| `undefined`\> The previous value, or `undefined` if the key did not exist. ###### See https://redis.io/commands/getdel/ ###### Inherited from `BasicKeyspace.getAndDelete` ##### getAndSet() ```ts getAndSet( key, value, options?): Promise; ``` Updates the value of key to val and returns the previously stored value. If the key does not already exist, it sets it and returns `undefined`. ###### Parameters ###### key `K` ###### value `string` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`string` \| `undefined`\> The previous value, or `undefined` if the key did not exist. ###### See https://redis.io/commands/getset/ ###### Inherited from `BasicKeyspace.getAndSet` ##### getRange() ```ts getRange( key, start, end): Promise; ``` Returns a substring of the string value stored at key. The `start` and `end` values are zero-based indices, but unlike typical slicing the `end` value is inclusive. Negative values can be used in order to provide an offset starting from the end of the string, so -1 means the last character. If the string does not exist it returns the empty string. ###### Parameters ###### key `K` The cache key. ###### start `number` Start index (inclusive, 0-based). ###### end `number` End index (inclusive, 0-based). Use -1 for end of string. ###### Returns `Promise`\<`string`\> The substring. ###### See https://redis.io/commands/getrange/ ##### len() `len(key): Promise` Returns the length of the string value stored at key. Non-existing keys are considered as empty strings. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number`\> The string length. ###### See https://redis.io/commands/strlen/ ##### mapKey() `protected mapKey(key): string` Maps a key to its Redis key string. ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `BasicKeyspace.mapKey` ##### multiGet() `multiGet(...keys): Promise<(string | undefined)[]>` Gets the values stored at multiple keys. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<(`string` \| `undefined`)[]\> An array of values in the same order as the provided keys. Each element is the value or `undefined` if the key was not found. ###### See https://redis.io/commands/mget/ ###### Inherited from `BasicKeyspace.multiGet` ##### multiSet() `multiSet(entries, options?): Promise` Updates the values stored at multiple keys. The keyspace's expiry (or the expiry provided in `options`) is applied to all keys. ###### Parameters ###### entries \[`K`, `string`\][] The key-value pairs to set. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/mset/ ###### Inherited from `BasicKeyspace.multiSet` ##### replace() ```ts replace( key, value, options?): Promise; ``` Replaces the existing value stored at key to val. ###### Parameters ###### key `K` ###### value `string` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the key does not already exist. ###### See https://redis.io/commands/set/ ###### Inherited from `BasicKeyspace.replace` ##### resolveTtl() `protected resolveTtl(options?): number | undefined` Resolves the TTL for a write operation. Returns i64 sentinel for NAPI: undefined=no config, -1=KeepTTL, -2=Persist/NeverExpire, >=0=ms ###### Parameters ###### options? [`WriteOptions`](#writeoptions) ###### Returns `number` \| `undefined` ###### Inherited from `BasicKeyspace.resolveTtl` ##### serialize() `protected serialize(value): Buffer` Serializes a value to a Buffer for storage. ###### Parameters ###### value `string` ###### Returns `Buffer` ###### Overrides `BasicKeyspace.serialize` ##### set() ```ts set( key, value, options?): Promise; ``` Updates the value stored at key to val. ###### Parameters ###### key `K` ###### value `string` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/set/ ###### Inherited from `BasicKeyspace.set` ##### setIfNotExists() ```ts setIfNotExists( key, value, options?): Promise; ``` Sets the value stored at key to val, but only if the key does not exist beforehand. ###### Parameters ###### key `K` ###### value `string` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the key already exists. ###### See https://redis.io/commands/setnx/ ###### Inherited from `BasicKeyspace.setIfNotExists` ##### setRange() ```ts setRange( key, offset, value, options?): Promise; ``` Overwrites part of the string stored at key, starting at the zero-based `offset` and for the entire length of `value`, extending the string if necessary. If the offset is larger than the current string length stored at key, the string is first padded with zero-bytes to make offset fit. Non-existing keys are considered as empty strings. ###### Parameters ###### key `K` The cache key. ###### offset `number` Zero-based byte offset to start writing at. ###### value `string` The string to write. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The length of the string after the operation. ###### See https://redis.io/commands/setrange/ ##### with() `with(options): this` Returns a shallow clone of this keyspace with the specified write options applied. This allows setting expiry for a chain of operations. ###### Parameters ###### options [`WriteOptions`](#writeoptions) ###### Returns `this` ###### Example `await myKeyspace.with({ expiry: expireIn(5000) }).set(key, value)` ###### Inherited from `BasicKeyspace.with` *** ### StringListKeyspace StringListKeyspace stores lists of string values. #### Example ```ts const recentViews = new StringListKeyspace(cluster, { keyPattern: "recent-views/:userId", defaultExpiry: ExpireIn(86400000), // 24 hours }); await recentViews.pushLeft("user1", "product-123", "product-456"); const views = await recentViews.items("user1"); ``` #### Extends - `ListKeyspace`\<`K`, `string`\> #### Type Parameters ##### K `K` #### Constructors ##### Constructor `new StringListKeyspace(cluster, config): StringListKeyspace` ###### Parameters ###### cluster [`CacheCluster`](#cachecluster) ###### config [`KeyspaceConfig`](#keyspaceconfig)\<`K`\> ###### Returns [`StringListKeyspace`](#stringlistkeyspace)\<`K`\> ###### Overrides `ListKeyspace.constructor` #### Properties ##### cluster `protected readonly cluster: CacheCluster` ###### Inherited from `ListKeyspace.cluster` ##### config `protected readonly config: KeyspaceConfig` ###### Inherited from `ListKeyspace.config` ##### keyMapper `protected readonly keyMapper: (key) => string` ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `ListKeyspace.keyMapper` #### Methods ##### delete() `delete(...keys): Promise` Deletes the specified keys. If a key does not exist it is ignored. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`number`\> The number of keys that were deleted. ###### See https://redis.io/commands/del/ ###### Inherited from `ListKeyspace.delete` ##### deserializeItem() `protected deserializeItem(data): string` ###### Parameters ###### data `Buffer` ###### Returns `string` ###### Overrides `ListKeyspace.deserializeItem` ##### get() `get(key, index): Promise` Returns the value of the list element at the given index. Negative indices can be used to indicate offsets from the end of the list, where -1 is the last element of the list, -2 the penultimate element, and so on. ###### Parameters ###### key `K` The cache key. ###### index `number` Zero-based index of the element to retrieve. ###### Returns `Promise`\<`string` \| `undefined`\> The value at the index, or `undefined` if out of range or the key does not exist. ###### See https://redis.io/commands/lindex/ ###### Inherited from `ListKeyspace.get` ##### getRange() ```ts getRange( key, start, stop): Promise; ``` Returns the elements in the list stored at key between `start` and `stop` (inclusive). Both are zero-based indices. Negative indices can be used to indicate offsets from the end of the list, where -1 is the last element of the list, -2 the penultimate element, and so on. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` The cache key. ###### start `number` Start index (inclusive). ###### stop `number` Stop index (inclusive). ###### Returns `Promise`\<`string`[]\> The elements in the specified range. ###### See https://redis.io/commands/lrange/ ###### Inherited from `ListKeyspace.getRange` ##### insertAfter() ```ts insertAfter( key, pivot, value, options?): Promise; ``` Inserts `value` into the list stored at key, at the position just after `pivot`. If the list does not contain `pivot`, the value is not inserted and -1 is returned. ###### Parameters ###### key `K` The cache key. ###### pivot `string` The existing element to insert after. ###### value `string` The value to insert. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new list length, or -1 if `pivot` was not found. ###### See https://redis.io/commands/linsert/ ###### Inherited from `ListKeyspace.insertAfter` ##### insertBefore() ```ts insertBefore( key, pivot, value, options?): Promise; ``` Inserts `value` into the list stored at key, at the position just before `pivot`. If the list does not contain `pivot`, the value is not inserted and -1 is returned. ###### Parameters ###### key `K` The cache key. ###### pivot `string` The existing element to insert before. ###### value `string` The value to insert. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The new list length, or -1 if `pivot` was not found. ###### See https://redis.io/commands/linsert/ ###### Inherited from `ListKeyspace.insertBefore` ##### items() `items(key): Promise` Returns all the elements in the list stored at key. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` ###### Returns `Promise`\<`string`[]\> All elements in the list. ###### See https://redis.io/commands/lrange/ ###### Inherited from `ListKeyspace.items` ##### len() `len(key): Promise` Returns the length of the list stored at key. Non-existing keys are considered as empty lists. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number`\> The list length. ###### See https://redis.io/commands/llen/ ###### Inherited from `ListKeyspace.len` ##### mapKey() `protected mapKey(key): string` Maps a key to its Redis key string. ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `ListKeyspace.mapKey` ##### move() ```ts move( src, dst, srcPos, dstPos, options?): Promise; ``` Atomically moves an element from the list stored at `src` to the list stored at `dst`. The value moved can be either the head (`srcPos === "left"`) or tail (`srcPos === "right"`) of the list at `src`. Similarly, the value can be placed either at the head (`dstPos === "left"`) or tail (`dstPos === "right"`) of the list at `dst`. If `src` and `dst` are the same list, the value is atomically rotated from one end to the other when `srcPos !== dstPos`, or if `srcPos === dstPos` nothing happens. ###### Parameters ###### src `K` Source list key. ###### dst `K` Destination list key. ###### srcPos [`ListPosition`](#listposition) Position to pop from in the source list. ###### dstPos [`ListPosition`](#listposition) Position to push to in the destination list. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`string` \| `undefined`\> The moved element, or `undefined` if the source list does not exist. ###### See https://redis.io/commands/lmove/ ###### Inherited from `ListKeyspace.move` ##### popLeft() `popLeft(key, options?): Promise` Pops a single element off the head of the list stored at key. ###### Parameters ###### key `K` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`string` \| `undefined`\> The popped value, or `undefined` if the key does not exist. ###### See https://redis.io/commands/lpop/ ###### Inherited from `ListKeyspace.popLeft` ##### popRight() `popRight(key, options?): Promise` Pops a single element off the tail of the list stored at key. ###### Parameters ###### key `K` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`string` \| `undefined`\> The popped value, or `undefined` if the key does not exist. ###### See https://redis.io/commands/rpop/ ###### Inherited from `ListKeyspace.popRight` ##### pushLeft() `pushLeft(key, ...values): Promise` Pushes one or more values at the head of the list stored at key. If the key does not already exist, it is first created as an empty list. If multiple values are given, they are inserted one after another, starting with the leftmost value. For instance, `pushLeft(key, "a", "b", "c")` will result in a list containing "c" as its first element, "b" as its second, and "a" as its third. ###### Parameters ###### key `K` ###### values ...`string`[] ###### Returns `Promise`\<`number`\> The length of the list after the operation. ###### See https://redis.io/commands/lpush/ ###### Inherited from `ListKeyspace.pushLeft` ##### pushRight() `pushRight(key, ...values): Promise` Pushes one or more values at the tail of the list stored at key. If the key does not already exist, it is first created as an empty list. If multiple values are given, they are inserted one after another, starting with the leftmost value. For instance, `pushRight(key, "a", "b", "c")` will result in a list containing "a" as its first element, "b" as its second, and "c" as its third. ###### Parameters ###### key `K` ###### values ...`string`[] ###### Returns `Promise`\<`number`\> The length of the list after the operation. ###### See https://redis.io/commands/rpush/ ###### Inherited from `ListKeyspace.pushRight` ##### removeAll() ```ts removeAll( key, value, options?): Promise; ``` Removes all occurrences of `value` in the list stored at key. If the list does not contain `value`, or the list does not exist, returns 0. ###### Parameters ###### key `K` The cache key. ###### value `string` The value to remove. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The number of elements removed. ###### See https://redis.io/commands/lrem/ ###### Inherited from `ListKeyspace.removeAll` ##### removeFirst() ```ts removeFirst( key, count, value, options?): Promise; ``` Removes the first `count` occurrences of `value` in the list stored at key, scanning from head to tail. If the list does not contain `value`, or the list does not exist, returns 0. ###### Parameters ###### key `K` The cache key. ###### count `number` Maximum number of occurrences to remove. ###### value `string` The value to remove. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The number of elements removed. ###### See https://redis.io/commands/lrem/ ###### Inherited from `ListKeyspace.removeFirst` ##### removeLast() ```ts removeLast( key, count, value, options?): Promise; ``` Removes the last `count` occurrences of `value` in the list stored at key, scanning from tail to head. If the list does not contain `value`, or the list does not exist, returns 0. ###### Parameters ###### key `K` The cache key. ###### count `number` Maximum number of occurrences to remove. ###### value `string` The value to remove. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`number`\> The number of elements removed. ###### See https://redis.io/commands/lrem/ ###### Inherited from `ListKeyspace.removeLast` ##### resolveTtl() `protected resolveTtl(options?): number | undefined` Resolves the TTL for a write operation. Returns i64 sentinel for NAPI: undefined=no config, -1=KeepTTL, -2=Persist/NeverExpire, >=0=ms ###### Parameters ###### options? [`WriteOptions`](#writeoptions) ###### Returns `number` \| `undefined` ###### Inherited from `ListKeyspace.resolveTtl` ##### serializeItem() `protected serializeItem(value): Buffer` ###### Parameters ###### value `string` ###### Returns `Buffer` ###### Overrides `ListKeyspace.serializeItem` ##### set() ```ts set( key, index, value, options?): Promise; ``` Updates the list element at the given index. Negative indices can be used to indicate offsets from the end of the list, where -1 is the last element of the list, -2 the penultimate element, and so on. ###### Parameters ###### key `K` The cache key. ###### index `number` Zero-based index of the element to update. ###### value `string` The new value. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the index is out of range. ###### See https://redis.io/commands/lset/ ###### Inherited from `ListKeyspace.set` ##### trim() ```ts trim( key, start, stop, options?): Promise; ``` Trims the list stored at key to only contain the elements between the indices `start` and `stop` (inclusive). Both are zero-based indices. Negative indices can be used to indicate offsets from the end of the list, where -1 is the last element of the list, -2 the penultimate element, and so on. Out of range indices are valid and are treated as if they specify the start or end of the list, respectively. If `start` > `stop` the end result is an empty list. ###### Parameters ###### key `K` The cache key. ###### start `number` Start index (inclusive). ###### stop `number` Stop index (inclusive). ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/ltrim/ ###### Inherited from `ListKeyspace.trim` ##### with() `with(options): this` Returns a shallow clone of this keyspace with the specified write options applied. This allows setting expiry for a chain of operations. ###### Parameters ###### options [`WriteOptions`](#writeoptions) ###### Returns `this` ###### Example `await myKeyspace.with({ expiry: expireIn(5000) }).set(key, value)` ###### Inherited from `ListKeyspace.with` *** ### StringSetKeyspace StringSetKeyspace stores sets of unique string values. #### Example ```ts const tags = new StringSetKeyspace(cluster, { keyPattern: "tags/:articleId", }); await tags.add("article1", "typescript", "programming", "web"); const hasTech = await tags.contains("article1", "typescript"); const allTags = await tags.items("article1"); const tagSet = await tags.itemsSet("article1"); ``` #### Extends - `SetKeyspace`\<`K`, `string`\> #### Type Parameters ##### K `K` #### Constructors ##### Constructor `new StringSetKeyspace(cluster, config): StringSetKeyspace` ###### Parameters ###### cluster [`CacheCluster`](#cachecluster) ###### config [`KeyspaceConfig`](#keyspaceconfig)\<`K`\> ###### Returns [`StringSetKeyspace`](#stringsetkeyspace)\<`K`\> ###### Overrides `SetKeyspace.constructor` #### Properties ##### cluster `protected readonly cluster: CacheCluster` ###### Inherited from `SetKeyspace.cluster` ##### config `protected readonly config: KeyspaceConfig` ###### Inherited from `SetKeyspace.config` ##### keyMapper `protected readonly keyMapper: (key) => string` ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `SetKeyspace.keyMapper` #### Methods ##### add() `add(key, ...members): Promise` Adds one or more values to the set stored at key. If the key does not already exist, it is first created as an empty set. ###### Parameters ###### key `K` ###### members ...`string`[] ###### Returns `Promise`\<`number`\> The number of values that were added to the set, not including values already present beforehand. ###### See https://redis.io/commands/sadd/ ###### Inherited from `SetKeyspace.add` ##### contains() `contains(key, member): Promise` Reports whether the set stored at key contains the given value. If the key does not exist it returns `false`. ###### Parameters ###### key `K` ###### member `string` ###### Returns `Promise`\<`boolean`\> `true` if the member exists in the set, `false` otherwise. ###### See https://redis.io/commands/sismember/ ###### Inherited from `SetKeyspace.contains` ##### delete() `delete(...keys): Promise` Deletes the specified keys. If a key does not exist it is ignored. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`number`\> The number of keys that were deleted. ###### See https://redis.io/commands/del/ ###### Inherited from `SetKeyspace.delete` ##### deserializeItem() `protected deserializeItem(data): string` ###### Parameters ###### data `Buffer` ###### Returns `string` ###### Overrides `SetKeyspace.deserializeItem` ##### diff() `diff(...keys): Promise` Computes the set difference between the first set and all the consecutive sets. Set difference means the values present in the first set that are not present in any of the other sets. Keys that do not exist are considered as empty sets. ###### Parameters ###### keys ...`K`[] Keys of sets to compute difference for. At least one must be provided. ###### Returns `Promise`\<`string`[]\> Members in the first set but not in any of the other sets. ###### Throws If no keys are provided. ###### See https://redis.io/commands/sdiff/ ###### Inherited from `SetKeyspace.diff` ##### diffSet() `diffSet(...keys): Promise>` Identical to [diff](#diff-1) except it returns the values as a `Set`. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`Set`\<`string`\>\> ###### See https://redis.io/commands/sdiff/ ###### Inherited from `SetKeyspace.diffSet` ##### diffStore() `diffStore(destination, ...keys): Promise` Computes the set difference between keys (like [diff](#diff-1)) and stores the result in `destination`. ###### Parameters ###### destination `K` Key to store the result. ###### keys ...`K`[] Keys of sets to compute difference for. ###### Returns `Promise`\<`number`\> The size of the resulting set. ###### See https://redis.io/commands/sdiffstore/ ###### Inherited from `SetKeyspace.diffStore` ##### intersect() `intersect(...keys): Promise` Computes the set intersection between the sets stored at the given keys. Set intersection means the values common to all the provided sets. Keys that do not exist are considered to be empty sets. As a result, if any key is missing the final result is the empty set. ###### Parameters ###### keys ...`K`[] Keys of sets to compute intersection for. At least one must be provided. ###### Returns `Promise`\<`string`[]\> Members common to all sets. ###### Throws If no keys are provided. ###### See https://redis.io/commands/sinter/ ###### Inherited from `SetKeyspace.intersect` ##### intersectSet() `intersectSet(...keys): Promise>` Identical to [intersect](#intersect-1) except it returns the values as a `Set`. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`Set`\<`string`\>\> ###### See https://redis.io/commands/sinter/ ###### Inherited from `SetKeyspace.intersectSet` ##### intersectStore() `intersectStore(destination, ...keys): Promise` Computes the set intersection between keys (like [intersect](#intersect-1)) and stores the result in `destination`. ###### Parameters ###### destination `K` Key to store the result. ###### keys ...`K`[] Keys of sets to compute intersection for. ###### Returns `Promise`\<`number`\> The size of the resulting set. ###### See https://redis.io/commands/sinterstore/ ###### Inherited from `SetKeyspace.intersectStore` ##### items() `items(key): Promise` Returns the elements in the set stored at key. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` ###### Returns `Promise`\<`string`[]\> All members of the set. ###### See https://redis.io/commands/smembers/ ###### Inherited from `SetKeyspace.items` ##### itemsSet() `itemsSet(key): Promise>` Identical to [items](#items-3) except it returns the values as a `Set`. If the key does not exist it returns an empty `Set`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`Set`\<`string`\>\> All members of the set as a `Set`. ###### See https://redis.io/commands/smembers/ ###### Inherited from `SetKeyspace.itemsSet` ##### len() `len(key): Promise` Returns the number of elements in the set stored at key. If the key does not exist it returns 0. ###### Parameters ###### key `K` ###### Returns `Promise`\<`number`\> The set cardinality. ###### See https://redis.io/commands/scard/ ###### Inherited from `SetKeyspace.len` ##### mapKey() `protected mapKey(key): string` Maps a key to its Redis key string. ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `SetKeyspace.mapKey` ##### move() ```ts move( src, dst, member, options?): Promise; ``` Atomically moves the given member from the set stored at `src` to the set stored at `dst`. If the element already exists in `dst` it is still removed from `src`. ###### Parameters ###### src `K` Source set key. ###### dst `K` Destination set key. ###### member `string` The member to move. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`boolean`\> `true` if the member was moved, `false` if not found in `src`. ###### See https://redis.io/commands/smove/ ###### Inherited from `SetKeyspace.move` ##### pop() ```ts pop( key, count, options?): Promise; ``` Removes up to `count` random elements (bounded by the set's size) from the set stored at key and returns them. If the set is empty it returns an empty array. ###### Parameters ###### key `K` The cache key. ###### count `number` Number of members to pop. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`string`[]\> The removed members (may be fewer than `count` if the set is small). ###### See https://redis.io/commands/spop/ ###### Inherited from `SetKeyspace.pop` ##### popOne() `popOne(key, options?): Promise` Removes a random element from the set stored at key and returns it. ###### Parameters ###### key `K` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`string` \| `undefined`\> The removed member, or `undefined` if the set is empty. ###### See https://redis.io/commands/spop/ ###### Inherited from `SetKeyspace.popOne` ##### remove() `remove(key, ...members): Promise` Removes one or more values from the set stored at key. Values not present in the set are ignored. If the key does not already exist, it is a no-op. ###### Parameters ###### key `K` ###### members ...`string`[] ###### Returns `Promise`\<`number`\> The number of values that were removed from the set. ###### See https://redis.io/commands/srem/ ###### Inherited from `SetKeyspace.remove` ##### resolveTtl() `protected resolveTtl(options?): number | undefined` Resolves the TTL for a write operation. Returns i64 sentinel for NAPI: undefined=no config, -1=KeepTTL, -2=Persist/NeverExpire, >=0=ms ###### Parameters ###### options? [`WriteOptions`](#writeoptions) ###### Returns `number` \| `undefined` ###### Inherited from `SetKeyspace.resolveTtl` ##### sample() `sample(key, count): Promise` Returns up to `count` distinct random elements from the set stored at key. The same element is never returned multiple times. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` The cache key. ###### count `number` Number of distinct members to return. ###### Returns `Promise`\<`string`[]\> Random members (may be fewer than `count` if the set is small). ###### See https://redis.io/commands/srandmember/ ###### Inherited from `SetKeyspace.sample` ##### sampleOne() `sampleOne(key): Promise` Returns a random member from the set stored at key without removing it. ###### Parameters ###### key `K` ###### Returns `Promise`\<`string` \| `undefined`\> A random member, or `undefined` if the key does not exist. ###### See https://redis.io/commands/srandmember/ ###### Inherited from `SetKeyspace.sampleOne` ##### sampleWithReplacement() `sampleWithReplacement(key, count): Promise` Returns `count` random elements from the set stored at key. The same element may be returned multiple times. If the key does not exist it returns an empty array. ###### Parameters ###### key `K` The cache key. ###### count `number` Number of members to return (may include duplicates). ###### Returns `Promise`\<`string`[]\> Random members, possibly with duplicates. ###### See https://redis.io/commands/srandmember/ ###### Inherited from `SetKeyspace.sampleWithReplacement` ##### serializeItem() `protected serializeItem(value): Buffer` ###### Parameters ###### value `string` ###### Returns `Buffer` ###### Overrides `SetKeyspace.serializeItem` ##### union() `union(...keys): Promise` Computes the set union between the sets stored at the given keys. Set union means the values present in at least one of the provided sets. Keys that do not exist are considered to be empty sets. ###### Parameters ###### keys ...`K`[] Keys of sets to compute union for. At least one must be provided. ###### Returns `Promise`\<`string`[]\> Members in any of the provided sets. ###### Throws If no keys are provided. ###### See https://redis.io/commands/sunion/ ###### Inherited from `SetKeyspace.union` ##### unionSet() `unionSet(...keys): Promise>` Identical to [union](#union-1) except it returns the values as a `Set`. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`Set`\<`string`\>\> ###### See https://redis.io/commands/sunion/ ###### Inherited from `SetKeyspace.unionSet` ##### unionStore() `unionStore(destination, ...keys): Promise` Computes the set union between sets (like [union](#union-1)) and stores the result in `destination`. ###### Parameters ###### destination `K` Key to store the result. ###### keys ...`K`[] Keys of sets to compute union for. ###### Returns `Promise`\<`number`\> The size of the resulting set. ###### See https://redis.io/commands/sunionstore/ ###### Inherited from `SetKeyspace.unionStore` ##### with() `with(options): this` Returns a shallow clone of this keyspace with the specified write options applied. This allows setting expiry for a chain of operations. ###### Parameters ###### options [`WriteOptions`](#writeoptions) ###### Returns `this` ###### Example `await myKeyspace.with({ expiry: expireIn(5000) }).set(key, value)` ###### Inherited from `SetKeyspace.with` *** ### StructKeyspace StructKeyspace stores arbitrary objects serialized as JSON. #### Example ```ts interface User { id: string; name: string; email: string; } const users = new StructKeyspace(cluster, { keyPattern: "user/:id", defaultExpiry: ExpireIn(3600000), }); await users.set("user1", { id: "user1", name: "Alice", email: "alice@example.com" }); const user = await users.get("user1"); ``` #### Extends - `BasicKeyspace`\<`K`, `V`\> #### Type Parameters ##### K `K` ##### V `V` #### Constructors ##### Constructor `new StructKeyspace(cluster, config): StructKeyspace` ###### Parameters ###### cluster [`CacheCluster`](#cachecluster) ###### config [`KeyspaceConfig`](#keyspaceconfig)\<`K`\> ###### Returns [`StructKeyspace`](#structkeyspace)\<`K`, `V`\> ###### Overrides `BasicKeyspace.constructor` #### Properties ##### cluster `protected readonly cluster: CacheCluster` ###### Inherited from `BasicKeyspace.cluster` ##### config `protected readonly config: KeyspaceConfig` ###### Inherited from `BasicKeyspace.config` ##### keyMapper `protected readonly keyMapper: (key) => string` ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `BasicKeyspace.keyMapper` #### Methods ##### delete() `delete(...keys): Promise` Deletes the specified keys. If a key does not exist it is ignored. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<`number`\> The number of keys that were deleted. ###### See https://redis.io/commands/del/ ###### Inherited from `BasicKeyspace.delete` ##### deserialize() `protected deserialize(data): V` Deserializes a Buffer from storage to a value. ###### Parameters ###### data `Buffer` ###### Returns `V` ###### Overrides `BasicKeyspace.deserialize` ##### get() `get(key): Promise` Gets the value stored at key. If the key does not exist, it returns `undefined`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`V` \| `undefined`\> The value, or `undefined` if the key does not exist. ###### See https://redis.io/commands/get/ ###### Inherited from `BasicKeyspace.get` ##### getAndDelete() `getAndDelete(key): Promise` Deletes the key and returns the previously stored value. If the key does not already exist, it returns `undefined`. ###### Parameters ###### key `K` ###### Returns `Promise`\<`V` \| `undefined`\> The previous value, or `undefined` if the key did not exist. ###### See https://redis.io/commands/getdel/ ###### Inherited from `BasicKeyspace.getAndDelete` ##### getAndSet() ```ts getAndSet( key, value, options?): Promise; ``` Updates the value of key to val and returns the previously stored value. If the key does not already exist, it sets it and returns `undefined`. ###### Parameters ###### key `K` ###### value `V` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`V` \| `undefined`\> The previous value, or `undefined` if the key did not exist. ###### See https://redis.io/commands/getset/ ###### Inherited from `BasicKeyspace.getAndSet` ##### mapKey() `protected mapKey(key): string` Maps a key to its Redis key string. ###### Parameters ###### key `K` ###### Returns `string` ###### Inherited from `BasicKeyspace.mapKey` ##### multiGet() `multiGet(...keys): Promise<(V | undefined)[]>` Gets the values stored at multiple keys. ###### Parameters ###### keys ...`K`[] ###### Returns `Promise`\<(`V` \| `undefined`)[]\> An array of values in the same order as the provided keys. Each element is the value or `undefined` if the key was not found. ###### See https://redis.io/commands/mget/ ###### Inherited from `BasicKeyspace.multiGet` ##### multiSet() `multiSet(entries, options?): Promise` Updates the values stored at multiple keys. The keyspace's expiry (or the expiry provided in `options`) is applied to all keys. ###### Parameters ###### entries \[`K`, `V`\][] The key-value pairs to set. ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/mset/ ###### Inherited from `BasicKeyspace.multiSet` ##### replace() ```ts replace( key, value, options?): Promise; ``` Replaces the existing value stored at key to val. ###### Parameters ###### key `K` ###### value `V` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the key does not already exist. ###### See https://redis.io/commands/set/ ###### Inherited from `BasicKeyspace.replace` ##### resolveTtl() `protected resolveTtl(options?): number | undefined` Resolves the TTL for a write operation. Returns i64 sentinel for NAPI: undefined=no config, -1=KeepTTL, -2=Persist/NeverExpire, >=0=ms ###### Parameters ###### options? [`WriteOptions`](#writeoptions) ###### Returns `number` \| `undefined` ###### Inherited from `BasicKeyspace.resolveTtl` ##### serialize() `protected serialize(value): Buffer` Serializes a value to a Buffer for storage. ###### Parameters ###### value `V` ###### Returns `Buffer` ###### Overrides `BasicKeyspace.serialize` ##### set() ```ts set( key, value, options?): Promise; ``` Updates the value stored at key to val. ###### Parameters ###### key `K` ###### value `V` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### See https://redis.io/commands/set/ ###### Inherited from `BasicKeyspace.set` ##### setIfNotExists() ```ts setIfNotExists( key, value, options?): Promise; ``` Sets the value stored at key to val, but only if the key does not exist beforehand. ###### Parameters ###### key `K` ###### value `V` ###### options? [`WriteOptions`](#writeoptions) ###### Returns `Promise`\<`void`\> ###### Throws If the key already exists. ###### See https://redis.io/commands/setnx/ ###### Inherited from `BasicKeyspace.setIfNotExists` ##### with() `with(options): this` Returns a shallow clone of this keyspace with the specified write options applied. This allows setting expiry for a chain of operations. ###### Parameters ###### options [`WriteOptions`](#writeoptions) ###### Returns `this` ###### Example `await myKeyspace.with({ expiry: expireIn(5000) }).set(key, value)` ###### Inherited from `BasicKeyspace.with` ## Interfaces ### CacheClusterConfig Configuration options for a cache cluster. #### Properties ##### evictionPolicy? `optional evictionPolicy?: EvictionPolicy` The eviction policy to use when the cache is full. Defaults to "allkeys-lru". *** ### KeyspaceConfig Configuration for a cache keyspace. #### Type Parameters ##### K `K` #### Properties ##### defaultExpiry? `optional defaultExpiry?: Expiry` Default expiry for cache entries in this keyspace. If not set, entries do not expire. ##### keyPattern `keyPattern: string` The pattern for generating cache keys. Use `:fieldName` to include a field from the key type. ###### Example ```ts // For a simple key type (string, number) keyPattern: "user/:id" // For a struct key type keyPattern: "user/:userId/region/:region" ``` *** ### WriteOptions Options for write operations. #### Properties ##### expiry? `optional expiry?: Expiry` Expiry for this specific write operation. Overrides the keyspace's defaultExpiry. ## Type Aliases ### EvictionPolicy ```ts type EvictionPolicy = | "noeviction" | "allkeys-lru" | "allkeys-lfu" | "allkeys-random" | "volatile-lru" | "volatile-lfu" | "volatile-ttl" | "volatile-random"; ``` Redis eviction policy that determines how keys are evicted when memory is full. *** ### Expiry ```ts type Expiry = | { durationMs: number; type: "duration"; } | { hours: number; minutes: number; seconds: number; type: "time"; } | "never" | "keep-ttl"; ``` Expiry represents a cache key expiration configuration. Use the helper functions to create expiry configurations. *** ### ListPosition `type ListPosition = "left" | "right"` Position in a list (left/head or right/tail). ## Variables ### keepTTL `const keepTTL: Expiry = "keep-ttl"` keepTTL preserves the existing TTL when updating a cache entry. If the key doesn't exist, no TTL is set. *** ### neverExpire `const neverExpire: Expiry = "never"` neverExpire sets the cache entry to never expire. Note: Redis may still evict the key based on the eviction policy. ## Functions ### expireDailyAt() ```ts function expireDailyAt( hours, minutes, seconds): Expiry; ``` expireDailyAt sets the cache entry to expire at a specific time each day (UTC). #### Parameters ##### hours `number` Hour (0-23) ##### minutes `number` Minutes (0-59) ##### seconds `number` Seconds (0-59) #### Returns [`Expiry`](#expiry-1) *** ### expireIn() `function expireIn(ms): Expiry` expireIn sets the cache entry to expire after the specified duration. #### Parameters ##### ms `number` Duration in milliseconds #### Returns [`Expiry`](#expiry-1) *** ### expireInHours() `function expireInHours(hours): Expiry` expireInHours sets the cache entry to expire after the specified hours. #### Parameters ##### hours `number` Duration in hours #### Returns [`Expiry`](#expiry-1) *** ### expireInMinutes() `function expireInMinutes(minutes): Expiry` expireInMinutes sets the cache entry to expire after the specified minutes. #### Parameters ##### minutes `number` Duration in minutes #### Returns [`Expiry`](#expiry-1) *** ### expireInSeconds() `function expireInSeconds(seconds): Expiry` expireInSeconds sets the cache entry to expire after the specified seconds. #### Parameters ##### seconds `number` Duration in seconds #### Returns [`Expiry`](#expiry-1) --- Source: https://encore.dev/docs/ts/runtime/storage-objects Section: Building (Encore.ts) # encore.dev/storage/objects ## Classes ### Bucket Defines a new Object Storage bucket infrastructure resource. #### Extends - [`BucketPerms`](#bucketperms) #### Implements - [`Uploader`](#uploader) - [`SignedUploader`](#signeduploader) - [`Downloader`](#downloader) - [`SignedDownloader`](#signeddownloader) - [`Attrser`](#attrser) - [`Lister`](#lister) - [`Remover`](#remover) - [`PublicUrler`](#publicurler) #### Constructors ##### Constructor `new Bucket(name, cfg?): Bucket` Creates a new bucket with the given name and configuration ###### Parameters ###### name `string` ###### cfg? [`BucketConfig`](#bucketconfig) ###### Returns [`Bucket`](#bucket) ###### Overrides `BucketPerms.constructor` #### Properties ##### impl `impl: Bucket` #### Methods ##### attrs() `attrs(name, options?): Promise` Returns the object's attributes. Throws an error if the object does not exist. ###### Parameters ###### name `string` ###### options? [`AttrsOptions`](#attrsoptions) ###### Returns `Promise`\<[`ObjectAttrs`](#objectattrs)\> ###### Implementation of [`Attrser`](#attrser).[`attrs`](#attrs-1) ##### download() `download(name, options?): Promise` Downloads an object from the bucket and returns its contents. ###### Parameters ###### name `string` ###### options? [`DownloadOptions`](#downloadoptions) ###### Returns `Promise`\<`Buffer`\> ###### Implementation of [`Downloader`](#downloader).[`download`](#download-1) ##### exists() `exists(name, options?): Promise` Returns whether the object exists in the bucket. Throws an error on network failure. ###### Parameters ###### name `string` ###### options? [`ExistsOptions`](#existsoptions) ###### Returns `Promise`\<`boolean`\> ###### Implementation of [`Attrser`](#attrser).[`exists`](#exists-1) ##### list() `list(options): AsyncGenerator` ###### Parameters ###### options [`ListOptions`](#listoptions) ###### Returns `AsyncGenerator`\<[`ListEntry`](#listentry)\> ###### Implementation of [`Lister`](#lister).[`list`](#list-1) ##### publicUrl() `publicUrl(name): string` Returns the public URL for accessing the object with the given name. Throws an error if the bucket is not public. ###### Parameters ###### name `string` ###### Returns `string` ###### Implementation of [`PublicUrler`](#publicurler).[`publicUrl`](#publicurl-1) ##### ref() `ref

(): P` ###### Type Parameters ###### P `P` *extends* [`BucketPerms`](#bucketperms) ###### Returns `P` ##### remove() `remove(name, options?): Promise` Removes an object from the bucket. Throws an error on network failure. ###### Parameters ###### name `string` ###### options? [`DeleteOptions`](#deleteoptions) ###### Returns `Promise`\<`void`\> ###### Implementation of [`Remover`](#remover).[`remove`](#remove-1) ##### signedDownloadUrl() `signedDownloadUrl(name, options?): Promise` Generate an external URL to allow downloading an object from the bucket. Anyone with possession of the URL can download the given object without any additional auth. ###### Parameters ###### name `string` ###### options? [`DownloadUrlOptions`](#downloadurloptions) ###### Returns `Promise`\<[`SignedDownloadUrl`](#signeddownloadurl-2)\> ###### Implementation of [`SignedDownloader`](#signeddownloader).[`signedDownloadUrl`](#signeddownloadurl-1) ##### signedUploadUrl() `signedUploadUrl(name, options?): Promise` Generate an external URL to allow uploading an object to the bucket. Anyone with possession of the URL can write to the given object name without any additional auth. ###### Parameters ###### name `string` ###### options? [`UploadUrlOptions`](#uploadurloptions) ###### Returns `Promise`\<[`SignedUploadUrl`](#signeduploadurl-2)\> ###### Implementation of [`SignedUploader`](#signeduploader).[`signedUploadUrl`](#signeduploadurl-1) ##### upload() ```ts upload( name, data, options?): Promise; ``` Uploads an object to the bucket. ###### Parameters ###### name `string` ###### data `Buffer` ###### options? [`UploadOptions`](#uploadoptions) ###### Returns `Promise`\<[`ObjectAttrs`](#objectattrs)\> ###### Implementation of [`Uploader`](#uploader).[`upload`](#upload-1) ##### named() `static named(name): Bucket` Reference an existing bucket by name. To create a new storage bucket, use `new StorageBucket(...)` instead. ###### Type Parameters ###### name `name` *extends* `string` ###### Parameters ###### name `StringLiteral`\<`name`\> ###### Returns [`Bucket`](#bucket) *** ### ObjectNotFound #### Extends - [`ObjectsError`](#objectserror) #### Constructors ##### Constructor `new ObjectNotFound(msg): ObjectNotFound` ###### Parameters ###### msg `string` ###### Returns [`ObjectNotFound`](#objectnotfound) ###### Overrides [`ObjectsError`](#objectserror).[`constructor`](#constructor-2) *** ### ObjectsError #### Extends - `Error` #### Extended by - [`ObjectNotFound`](#objectnotfound) - [`PreconditionFailed`](#preconditionfailed) #### Constructors ##### Constructor `new ObjectsError(msg): ObjectsError` ###### Parameters ###### msg `string` ###### Returns [`ObjectsError`](#objectserror) ###### Overrides `Error.constructor` *** ### PreconditionFailed #### Extends - [`ObjectsError`](#objectserror) #### Constructors ##### Constructor `new PreconditionFailed(msg): PreconditionFailed` ###### Parameters ###### msg `string` ###### Returns [`PreconditionFailed`](#preconditionfailed) ###### Overrides [`ObjectsError`](#objectserror).[`constructor`](#constructor-2) ## Interfaces ### Attrser #### Extends - [`BucketPerms`](#bucketperms) #### Methods ##### attrs() `abstract attrs(name, options?): Promise` ###### Parameters ###### name `string` ###### options? [`AttrsOptions`](#attrsoptions) ###### Returns `Promise`\<[`ObjectAttrs`](#objectattrs)\> ##### exists() `abstract exists(name, options?): Promise` ###### Parameters ###### name `string` ###### options? [`ExistsOptions`](#existsoptions) ###### Returns `Promise`\<`boolean`\> *** ### AttrsOptions Options for retrieving the attributes of an object. #### Properties ##### version? `optional version?: string` The object version to retrieve attributes for. Defaults to the lastest version if unset. If bucket versioning is not enabled, this option is ignored. *** ### BucketConfig Configuration options for declaring a Bucket. #### Properties ##### public? `optional public?: boolean` Whether the objects in the bucket should be publicly accessible, via CDN. Defaults to false if unset. ##### versioned? `optional versioned?: boolean` Whether to enable versioning of the objects in the bucket. Defaults to false if unset. *** ### BucketPerms #### Extended by - [`Bucket`](#bucket) - [`Uploader`](#uploader) - [`SignedUploader`](#signeduploader) - [`Downloader`](#downloader) - [`SignedDownloader`](#signeddownloader) - [`Attrser`](#attrser) - [`Lister`](#lister) - [`PublicUrler`](#publicurler) - [`Remover`](#remover) *** ### DeleteOptions Options for deleting an object from a bucket. #### Properties ##### version? `optional version?: string` The object version to delete. Defaults to the lastest version if unset. If bucket versioning is not enabled, this option is ignored. *** ### Downloader #### Extends - [`BucketPerms`](#bucketperms) #### Methods ##### download() `abstract download(name, options?): Promise` ###### Parameters ###### name `string` ###### options? [`DownloadOptions`](#downloadoptions) ###### Returns `Promise`\<`Buffer`\> *** ### DownloadOptions Options for downloading an object from a bucket. #### Properties ##### version? `optional version?: string` The object version to download. Defaults to the lastest version if unset. If bucket versioning is not enabled, this option is ignored. *** ### DownloadUrlOptions Options for generating a signed download URL. #### Properties ##### ttl? `optional ttl?: number` The expiration time of the url, in seconds from signing. The maximum value is seven days. If no value is given, a default of one hour is used. *** ### ExistsOptions Options for checking the existence of an object. #### Properties ##### version? `optional version?: string` The object version to check for existence. Defaults to the lastest version if unset. If bucket versioning is not enabled, this option is ignored. *** ### ListEntry A single entry returned when listing objects in a bucket. #### Properties ##### etag `etag: string` ##### name `name: string` ##### size `size: number` *** ### Lister #### Extends - [`BucketPerms`](#bucketperms) #### Methods ##### list() `abstract list(options): AsyncGenerator` ###### Parameters ###### options [`ListOptions`](#listoptions) ###### Returns `AsyncGenerator`\<[`ListEntry`](#listentry)\> *** ### ListOptions Options for listing objects in a bucket. #### Properties ##### limit? `optional limit?: number` Maximum number of objects to return. Defaults to no limit. ##### prefix? `optional prefix?: string` Only include objects with this prefix in the listing. If unset, all objects are included. *** ### ObjectAttrs Describes the attributes of an object stored in a bucket. #### Properties ##### contentType? `optional contentType?: string` ##### etag `etag: string` ##### name `name: string` ##### size `size: number` ##### version? `optional version?: string` The version of the object, if bucket versioning is enabled. *** ### PublicUrler #### Extends - [`BucketPerms`](#bucketperms) #### Methods ##### publicUrl() `abstract publicUrl(name): string` ###### Parameters ###### name `string` ###### Returns `string` *** ### Remover #### Extends - [`BucketPerms`](#bucketperms) #### Methods ##### remove() `abstract remove(name, options?): Promise` ###### Parameters ###### name `string` ###### options? [`DeleteOptions`](#deleteoptions) ###### Returns `Promise`\<`void`\> *** ### SignedDownloader #### Extends - [`BucketPerms`](#bucketperms) #### Methods ##### signedDownloadUrl() `abstract signedDownloadUrl(name, options?): Promise` ###### Parameters ###### name `string` ###### options? [`DownloadUrlOptions`](#downloadurloptions) ###### Returns `Promise`\<[`SignedDownloadUrl`](#signeddownloadurl-2)\> *** ### SignedDownloadUrl A signed URL that allows downloading an object without additional auth. #### Properties ##### url `url: string` *** ### SignedUploader #### Extends - [`BucketPerms`](#bucketperms) #### Methods ##### signedUploadUrl() `abstract signedUploadUrl(name, options?): Promise` ###### Parameters ###### name `string` ###### options? [`UploadUrlOptions`](#uploadurloptions) ###### Returns `Promise`\<[`SignedUploadUrl`](#signeduploadurl-2)\> *** ### SignedUploadUrl A signed URL that allows uploading an object without additional auth. #### Properties ##### url `url: string` *** ### Uploader #### Extends - [`BucketPerms`](#bucketperms) #### Methods ##### upload() ```ts abstract upload( name, data, options?): Promise; ``` ###### Parameters ###### name `string` ###### data `Buffer` ###### options? [`UploadOptions`](#uploadoptions) ###### Returns `Promise`\<[`ObjectAttrs`](#objectattrs)\> *** ### UploadOptions Options for uploading an object to a bucket. #### Properties ##### contentType? `optional contentType?: string` ##### preconditions? ```ts optional preconditions?: { notExists?: boolean; }; ``` ###### notExists? `optional notExists?: boolean` *** ### UploadUrlOptions Options for generating a signed upload URL. #### Properties ##### ttl? `optional ttl?: number` The expiration time of the url, in seconds from signing. The maximum value is seven days. If no value is given, a default of one hour is used. ## Type Aliases ### ReadWriter `type ReadWriter = Uploader & SignedUploader & Downloader & SignedDownloader & Attrser & Lister & Remover` --- Source: https://encore.dev/docs/ts/runtime/storage-sqldb Section: Building (Encore.ts) # encore.dev/storage/sqldb ## Classes ### Connection Represents a dedicated connection to a database. #### Extends - `BaseQueryExecutor` #### Constructors ##### Constructor `new Connection(impl): Connection` ###### Parameters ###### impl `SQLConn` ###### Returns [`Connection`](#connection) ###### Overrides `BaseQueryExecutor.constructor` #### Properties ##### impl `protected readonly impl: SQLConn` ###### Overrides `BaseQueryExecutor.impl` #### Methods ##### close() `close(): Promise` Returns the connection to the database pool. ###### Returns `Promise`\<`void`\> ##### exec() `exec(strings, ...params): Promise` exec executes a query without returning any rows. ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`void`\> ###### Example ```ts const email = "foo@example.com"; const result = database.exec`DELETE FROM users WHERE email=${email}` ``` ###### Inherited from `BaseQueryExecutor.exec` ##### query() `query(strings, ...params): AsyncGenerator` query queries the database using a template string, replacing your placeholders in the template with parametrised values without risking SQL injections. It returns an async generator, that allows iterating over the results in a streaming fashion using `for await`. A `for await` loop closes the query however it ends. If you call `next()` yourself, call `return()` when you stop before the last row: until then the query keeps its connection busy, and any other query on that connection (such as the next statement in a transaction) waits for it. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `AsyncGenerator`\<`T`\> ###### Example ```ts const email = "foo@example.com"; const result = database.query`SELECT id FROM users WHERE email=${email}` This produces the query: "SELECT id FROM users WHERE email=$1". ``` ###### Inherited from `BaseQueryExecutor.query` ##### queryAll() `queryAll(strings, ...params): Promise` queryAll queries the database using a template string, replacing your placeholders in the template with parametrised values without risking SQL injections. It returns an array of all results. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T`[]\> ###### Example ```ts const email = "foo@example.com"; const result = database.queryAll`SELECT id FROM users WHERE email=${email}` This produces the query: "SELECT id FROM users WHERE email=$1". ``` ###### Inherited from `BaseQueryExecutor.queryAll` ##### queryRow() `queryRow(strings, ...params): Promise` queryRow is like query but returns only a single row. If the query selects no rows it returns null. Otherwise it returns the first row and discards the rest. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T` \| `null`\> ###### Example ```ts const email = "foo@example.com"; const result = database.queryRow`SELECT id FROM users WHERE email=${email}` ``` ###### Inherited from `BaseQueryExecutor.queryRow` ##### rawExec() `rawExec(query, ...params): Promise` rawExec executes a query without returning any rows. ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `Promise`\<`void`\> A promise that resolves when the query has been executed. ###### Example ```ts const query = "DELETE FROM users WHERE email=$1"; const email = "foo@example.com"; await database.rawExec(query, email); ``` ###### Inherited from `BaseQueryExecutor.rawExec` ##### rawQuery() `rawQuery(query, ...params): AsyncGenerator` rawQuery queries the database using a raw parametrised SQL query and parameters. It returns an async generator, that allows iterating over the results in a streaming fashion using `for await`. A `for await` loop closes the query however it ends. If you call `next()` yourself, call `return()` when you stop before the last row: until then the query keeps its connection busy, and any other query on that connection (such as the next statement in a transaction) waits for it. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `AsyncGenerator`\<`T`\> An async generator that yields rows from the query result. ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; for await (const row of database.rawQuery(query, email)) { console.log(row); } ``` ###### Inherited from `BaseQueryExecutor.rawQuery` ##### rawQueryAll() `rawQueryAll(query, ...params): Promise` rawQueryAll queries the database using a raw parametrised SQL query and parameters. It returns an array of all results. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T`[]\> ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; const rows = await database.rawQueryAll(query, email); ``` ###### Inherited from `BaseQueryExecutor.rawQueryAll` ##### rawQueryRow() `rawQueryRow(query, ...params): Promise` rawQueryRow is like rawQuery but returns only a single row. If the query selects no rows, it returns null. Otherwise, it returns the first row and discards the rest. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `Promise`\<`T` \| `null`\> A promise that resolves to a single row or null. ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; const result = await database.rawQueryRow(query, email); console.log(result); ``` ###### Inherited from `BaseQueryExecutor.rawQueryRow` *** ### SQLDatabase Constructing a new database object will result in Encore provisioning a database with that name and returning this object to represent it. If you want to reference an existing database, use `Database.Named(name)` as it is a compile error to create duplicate databases. #### Extends - `BaseQueryExecutor` #### Constructors ##### Constructor `new SQLDatabase(name, cfg?): SQLDatabase` ###### Parameters ###### name `string` ###### cfg? [`SQLDatabaseConfig`](#sqldatabaseconfig) ###### Returns [`SQLDatabase`](#sqldatabase) ###### Overrides `BaseQueryExecutor.constructor` #### Properties ##### impl `protected readonly impl: SQLDatabase` ###### Overrides `BaseQueryExecutor.impl` #### Accessors ##### connectionString ###### Get Signature `get connectionString(): string` Returns the connection string for the database ###### Returns `string` #### Methods ##### acquire() `acquire(): Promise` Acquires a database connection from the database pool. When the connection is closed or is garbage-collected, it is returned to the pool. ###### Returns `Promise`\<[`Connection`](#connection)\> a new connection to the database ##### begin() `begin(): Promise` Begins a database transaction. Prefer the `await using` pattern, which automatically rolls back the transaction if neither `commit` nor `rollback` is called before the variable goes out of scope: ```ts await using tx = await db.begin(); await tx.exec`INSERT INTO ...`; await tx.commit(); ``` If you can't use `await using`, make sure to always call `commit` or `rollback` yourself to prevent hanging transactions. ###### Returns `Promise`\<[`Transaction`](#transaction)\> a transaction object that implements `AsyncDisposable` ##### exec() `exec(strings, ...params): Promise` exec executes a query without returning any rows. ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`void`\> ###### Example ```ts const email = "foo@example.com"; const result = database.exec`DELETE FROM users WHERE email=${email}` ``` ###### Inherited from `BaseQueryExecutor.exec` ##### query() `query(strings, ...params): AsyncGenerator` query queries the database using a template string, replacing your placeholders in the template with parametrised values without risking SQL injections. It returns an async generator, that allows iterating over the results in a streaming fashion using `for await`. A `for await` loop closes the query however it ends. If you call `next()` yourself, call `return()` when you stop before the last row: until then the query keeps its connection busy, and any other query on that connection (such as the next statement in a transaction) waits for it. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `AsyncGenerator`\<`T`\> ###### Example ```ts const email = "foo@example.com"; const result = database.query`SELECT id FROM users WHERE email=${email}` This produces the query: "SELECT id FROM users WHERE email=$1". ``` ###### Inherited from `BaseQueryExecutor.query` ##### queryAll() `queryAll(strings, ...params): Promise` queryAll queries the database using a template string, replacing your placeholders in the template with parametrised values without risking SQL injections. It returns an array of all results. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T`[]\> ###### Example ```ts const email = "foo@example.com"; const result = database.queryAll`SELECT id FROM users WHERE email=${email}` This produces the query: "SELECT id FROM users WHERE email=$1". ``` ###### Inherited from `BaseQueryExecutor.queryAll` ##### queryRow() `queryRow(strings, ...params): Promise` queryRow is like query but returns only a single row. If the query selects no rows it returns null. Otherwise it returns the first row and discards the rest. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T` \| `null`\> ###### Example ```ts const email = "foo@example.com"; const result = database.queryRow`SELECT id FROM users WHERE email=${email}` ``` ###### Inherited from `BaseQueryExecutor.queryRow` ##### rawExec() `rawExec(query, ...params): Promise` rawExec executes a query without returning any rows. ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `Promise`\<`void`\> A promise that resolves when the query has been executed. ###### Example ```ts const query = "DELETE FROM users WHERE email=$1"; const email = "foo@example.com"; await database.rawExec(query, email); ``` ###### Inherited from `BaseQueryExecutor.rawExec` ##### rawQuery() `rawQuery(query, ...params): AsyncGenerator` rawQuery queries the database using a raw parametrised SQL query and parameters. It returns an async generator, that allows iterating over the results in a streaming fashion using `for await`. A `for await` loop closes the query however it ends. If you call `next()` yourself, call `return()` when you stop before the last row: until then the query keeps its connection busy, and any other query on that connection (such as the next statement in a transaction) waits for it. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `AsyncGenerator`\<`T`\> An async generator that yields rows from the query result. ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; for await (const row of database.rawQuery(query, email)) { console.log(row); } ``` ###### Inherited from `BaseQueryExecutor.rawQuery` ##### rawQueryAll() `rawQueryAll(query, ...params): Promise` rawQueryAll queries the database using a raw parametrised SQL query and parameters. It returns an array of all results. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T`[]\> ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; const rows = await database.rawQueryAll(query, email); ``` ###### Inherited from `BaseQueryExecutor.rawQueryAll` ##### rawQueryRow() `rawQueryRow(query, ...params): Promise` rawQueryRow is like rawQuery but returns only a single row. If the query selects no rows, it returns null. Otherwise, it returns the first row and discards the rest. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `Promise`\<`T` \| `null`\> A promise that resolves to a single row or null. ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; const result = await database.rawQueryRow(query, email); console.log(result); ``` ###### Inherited from `BaseQueryExecutor.rawQueryRow` ##### named() `static named(name): SQLDatabase` Reference an existing database by name, if the database doesn't exist yet, use `new Database(name)` instead. ###### Type Parameters ###### name `name` *extends* `string` ###### Parameters ###### name `StringLiteral`\<`name`\> ###### Returns [`SQLDatabase`](#sqldatabase) *** ### Transaction Represents a database transaction. `Transaction` implements `AsyncDisposable`, so the recommended usage is the `await using` pattern — it automatically rolls back the transaction if neither `commit` nor `rollback` is called before the variable goes out of scope: ```ts await using tx = await db.begin(); await tx.exec`INSERT INTO ...`; await tx.commit(); ``` If you can't use `await using`, make sure to always call `commit` or `rollback` yourself to prevent hanging transactions. #### Extends - `BaseQueryExecutor` #### Implements - `AsyncDisposable` #### Constructors ##### Constructor `new Transaction(impl): Transaction` ###### Parameters ###### impl `Transaction` ###### Returns [`Transaction`](#transaction) ###### Overrides `BaseQueryExecutor.constructor` #### Properties ##### impl `protected readonly impl: Transaction` ###### Overrides `BaseQueryExecutor.impl` #### Methods ##### \[asyncDispose\]() `asyncDispose: Promise` ###### Returns `Promise`\<`void`\> ###### Implementation of `AsyncDisposable.[asyncDispose]` ##### commit() `commit(): Promise` Commit the transaction. ###### Returns `Promise`\<`void`\> ##### exec() `exec(strings, ...params): Promise` exec executes a query without returning any rows. ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`void`\> ###### Example ```ts const email = "foo@example.com"; const result = database.exec`DELETE FROM users WHERE email=${email}` ``` ###### Inherited from `BaseQueryExecutor.exec` ##### query() `query(strings, ...params): AsyncGenerator` query queries the database using a template string, replacing your placeholders in the template with parametrised values without risking SQL injections. It returns an async generator, that allows iterating over the results in a streaming fashion using `for await`. A `for await` loop closes the query however it ends. If you call `next()` yourself, call `return()` when you stop before the last row: until then the query keeps its connection busy, and any other query on that connection (such as the next statement in a transaction) waits for it. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `AsyncGenerator`\<`T`\> ###### Example ```ts const email = "foo@example.com"; const result = database.query`SELECT id FROM users WHERE email=${email}` This produces the query: "SELECT id FROM users WHERE email=$1". ``` ###### Inherited from `BaseQueryExecutor.query` ##### queryAll() `queryAll(strings, ...params): Promise` queryAll queries the database using a template string, replacing your placeholders in the template with parametrised values without risking SQL injections. It returns an array of all results. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T`[]\> ###### Example ```ts const email = "foo@example.com"; const result = database.queryAll`SELECT id FROM users WHERE email=${email}` This produces the query: "SELECT id FROM users WHERE email=$1". ``` ###### Inherited from `BaseQueryExecutor.queryAll` ##### queryRow() `queryRow(strings, ...params): Promise` queryRow is like query but returns only a single row. If the query selects no rows it returns null. Otherwise it returns the first row and discards the rest. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### strings `TemplateStringsArray` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T` \| `null`\> ###### Example ```ts const email = "foo@example.com"; const result = database.queryRow`SELECT id FROM users WHERE email=${email}` ``` ###### Inherited from `BaseQueryExecutor.queryRow` ##### rawExec() `rawExec(query, ...params): Promise` rawExec executes a query without returning any rows. ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `Promise`\<`void`\> A promise that resolves when the query has been executed. ###### Example ```ts const query = "DELETE FROM users WHERE email=$1"; const email = "foo@example.com"; await database.rawExec(query, email); ``` ###### Inherited from `BaseQueryExecutor.rawExec` ##### rawQuery() `rawQuery(query, ...params): AsyncGenerator` rawQuery queries the database using a raw parametrised SQL query and parameters. It returns an async generator, that allows iterating over the results in a streaming fashion using `for await`. A `for await` loop closes the query however it ends. If you call `next()` yourself, call `return()` when you stop before the last row: until then the query keeps its connection busy, and any other query on that connection (such as the next statement in a transaction) waits for it. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `AsyncGenerator`\<`T`\> An async generator that yields rows from the query result. ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; for await (const row of database.rawQuery(query, email)) { console.log(row); } ``` ###### Inherited from `BaseQueryExecutor.rawQuery` ##### rawQueryAll() `rawQueryAll(query, ...params): Promise` rawQueryAll queries the database using a raw parametrised SQL query and parameters. It returns an array of all results. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` ###### params ...[`Primitive`](#primitive)[] ###### Returns `Promise`\<`T`[]\> ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; const rows = await database.rawQueryAll(query, email); ``` ###### Inherited from `BaseQueryExecutor.rawQueryAll` ##### rawQueryRow() `rawQueryRow(query, ...params): Promise` rawQueryRow is like rawQuery but returns only a single row. If the query selects no rows, it returns null. Otherwise, it returns the first row and discards the rest. ###### Type Parameters ###### T `T` *extends* [`ResultRow`](#resultrow) = `Record`\<`string`, `any`\> ###### Parameters ###### query `string` The raw SQL query string. ###### params ...[`Primitive`](#primitive)[] The parameters to be used in the query. ###### Returns `Promise`\<`T` \| `null`\> A promise that resolves to a single row or null. ###### Example ```ts const query = "SELECT id FROM users WHERE email=$1"; const email = "foo@example.com"; const result = await database.rawQueryRow(query, email); console.log(result); ``` ###### Inherited from `BaseQueryExecutor.rawQueryRow` ##### rollback() `rollback(): Promise` Rollback the transaction. ###### Returns `Promise`\<`void`\> ## Interfaces ### SQLDatabaseConfig Configuration for a `SQLDatabase`. #### Properties ##### migrations? `optional migrations?: string | SQLMigrationsConfig` *** ### SQLMigrationsConfig Configures how database migrations are managed for a `SQLDatabase`. #### Properties ##### path `path: string` ##### source? `optional source?: "prisma" | "drizzle" | "drizzle/v1"` ## Type Aliases ### Primitive ```ts type Primitive = | string | string[] | number | number[] | boolean | boolean[] | Buffer | Date | Date[] | Record | Record[] | BigInt | BigInt[] | null | undefined; ``` Represents a type that can be used in query template literals *** ### ResultRow `type ResultRow = Record` Represents a single row from a query result --- Source: https://encore.dev/docs/ts/runtime/types Section: Building (Encore.ts) # encore.dev/types ## Classes ### Decimal A decimal type that can hold values with arbitrary precision. Unlike JavaScript's native number type, this can accurately represent decimal values without floating-point precision errors. #### Constructors ##### Constructor `new Decimal(value): Decimal` ###### Parameters ###### value [`ToDecimal`](#todecimal) ###### Returns [`Decimal`](#decimal) #### Accessors ##### value ###### Get Signature `get value(): string` ###### Returns `string` #### Methods ##### \[toPrimitive\]() `toPrimitive: string | number` ###### Parameters ###### hint `string` ###### Returns `string` \| `number` ##### add() `add(d): Decimal` Adds this decimal to another decimal value. ###### Parameters ###### d [`Decimal`](#decimal) \| [`ToDecimal`](#todecimal) ###### Returns [`Decimal`](#decimal) ##### div() `div(d): Decimal` Divides this decimal by another decimal value. ###### Parameters ###### d [`Decimal`](#decimal) \| [`ToDecimal`](#todecimal) ###### Returns [`Decimal`](#decimal) ##### mul() `mul(d): Decimal` Multiplies this decimal by another decimal value. ###### Parameters ###### d [`Decimal`](#decimal) \| [`ToDecimal`](#todecimal) ###### Returns [`Decimal`](#decimal) ##### sub() `sub(d): Decimal` Subtracts another decimal value from this decimal. ###### Parameters ###### d [`Decimal`](#decimal) \| [`ToDecimal`](#todecimal) ###### Returns [`Decimal`](#decimal) ##### toJSON() `toJSON(): string` ###### Returns `string` ##### toString() `toString(): string` ###### Returns `string` ## Type Aliases ### DurationString ```ts type DurationString = | durationComponent | `${durationComponent}${durationComponent}` | `${durationComponent} ${durationComponent}`; ``` A duration is a string representing a length of time. Examples: `"10s"`, `"500ms"`, `"5m"`, `"1h30m"`, `"1h 30m"`. *** ### ToDecimal `type ToDecimal = string | number | bigint` --- Source: https://encore.dev/docs/ts/runtime/validate Section: Building (Encore.ts) # encore.dev/validate ## Type Aliases ### EndsWith ```ts type EndsWith = { [___validate]?: { endsWith: S; }; }; ``` #### Type Parameters ##### S `S` *extends* `string` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { endsWith: S; }; ``` ###### endsWith `endsWith: S` *** ### IsEmail ```ts type IsEmail = { [___validate]?: { isEmail: true; }; }; ``` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { isEmail: true; }; ``` ###### isEmail `isEmail: true` *** ### IsURL ```ts type IsURL = { [___validate]?: { isURL: true; }; }; ``` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { isURL: true; }; ``` ###### isURL `isURL: true` *** ### MatchesRegexp ```ts type MatchesRegexp = { [___validate]?: { matchesRegexp: S; }; }; ``` #### Type Parameters ##### S `S` *extends* `string` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { matchesRegexp: S; }; ``` ###### matchesRegexp `matchesRegexp: S` *** ### Max ```ts type Max = { [___validate]?: { maxValue: N; }; }; ``` #### Type Parameters ##### N `N` *extends* `number` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { maxValue: N; }; ``` ###### maxValue `maxValue: N` *** ### MaxLen ```ts type MaxLen = { [___validate]?: { maxLen: N; }; }; ``` #### Type Parameters ##### N `N` *extends* `number` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { maxLen: N; }; ``` ###### maxLen `maxLen: N` *** ### Min ```ts type Min = { [___validate]?: { minValue: N; }; }; ``` #### Type Parameters ##### N `N` *extends* `number` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { minValue: N; }; ``` ###### minValue `minValue: N` *** ### MinLen ```ts type MinLen = { [___validate]?: { minLen: N; }; }; ``` #### Type Parameters ##### N `N` *extends* `number` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { minLen: N; }; ``` ###### minLen `minLen: N` *** ### StartsWith ```ts type StartsWith = { [___validate]?: { startsWith: S; }; }; ``` #### Type Parameters ##### S `S` *extends* `string` #### Properties ##### \[\_\_\_validate\]? ```ts optional [___validate]?: { startsWith: S; }; ``` ###### startsWith `startsWith: S` --- Source: https://encore.dev/docs/ts/develop/monorepo/turborepo Section: Building (Encore.ts) # Turborepo > Using Encore in a Turborepo monorepo [Turborepo](https://turbo.build/repo) is a build system for JavaScript and TypeScript monorepos. This guide shows how to set up an Encore application within a Turborepo monorepo that depends on shared packages requiring compilation. ## Overview When using Encore in a Turborepo monorepo, you may have shared packages (like utility libraries or shared types) that need to be built before the Encore app can use them. Since Encore parses your application on startup, these dependencies must be compiled first. This guide covers two scenarios: - **Local development**: Use Turborepo to build dependencies before running `encore run` - **Deployment**: Use Encore's `prebuild` hook to automatically build dependencies when deploying via Encore Cloud or exporting a Docker image ## Project structure A typical Turborepo setup with Encore looks like this: ``` my-turborepo/ ├── apps/ │ └── backend/ # Encore application │ ├── encore.app │ ├── package.json │ ├── tsconfig.json │ └── article/ │ └── article.ts ├── packages/ │ └── shared/ # Shared library requiring build │ ├── package.json │ ├── tsconfig.json │ ├── src/ │ │ └── index.ts │ └── dist/ # Built output │ └── index.js ├── turbo.json ├── package.json └── package-lock.json ``` ## Configuration ### Root package.json Configure npm workspaces to include your apps and packages: ```json -- package.json -- { "name": "my-turborepo", "private": true, "packageManager": "npm@10.0.0", "scripts": { "build": "turbo run build", "dev": "turbo run dev" }, "devDependencies": { "turbo": "^2.0.0", "typescript": "^5.0.0" }, "workspaces": [ "apps/*", "packages/*" ] } ``` The `packageManager` field is required by Turborepo. Adjust the version to match your installed npm version (run `npm --version` to check). ### turbo.json Configure Turborepo's build pipeline in the root `turbo.json`. The `@repo/backend#dev` task depends on the shared package being built first: ```json -- turbo.json -- { "$schema": "https://turbo.build/schema.json", "tasks": { "build": { "dependsOn": ["^build"], "outputs": ["dist/**"] }, "@repo/backend#dev": { "dependsOn": ["@repo/shared#build"], "cache": false, "persistent": true }, "dev": { "cache": false, "persistent": true } } } ``` The `@repo/backend#dev` task configuration ensures the shared package is built before running `encore run` in local development. ### Shared package Your shared package needs to compile TypeScript to JavaScript and expose the built output: **packages/shared/package.json:** ```json { "name": "@repo/shared", "version": "1.0.0", "private": true, "type": "module", "main": "./dist/index.js", "types": "./dist/index.d.ts", "exports": { ".": { "types": "./dist/index.d.ts", "default": "./dist/index.js" } }, "scripts": { "build": "tsc" }, "devDependencies": { "typescript": "^5.0.0" } } ``` **packages/shared/tsconfig.json:** ```json { "compilerOptions": { "outDir": "dist", "rootDir": "src", "moduleResolution": "bundler", "module": "ES2022", "target": "ES2022", "declaration": true }, "include": ["src"], "exclude": ["node_modules", "dist"] } ``` **packages/shared/src/index.ts:** ```ts // Types shared between frontend and backend export interface Article { slug: string; title: string; preview: string; } export interface CreateArticleRequest { title: string; content: string; } // Utility functions export function slugify(text: string): string { return text .toLowerCase() .trim() .replace(/[^\w\s-]/g, "") .replace(/\s+/g, "-"); } export function truncate(text: string, maxLength: number): string { if (text.length <= maxLength) return text; return text.slice(0, maxLength - 3) + "..."; } ``` ### Encore application The Encore app needs two key configurations: 1. **encore.app** - Use the `prebuild` hook to build dependencies during deployment 2. **package.json** - Declare the dependency on the shared package To create the Encore app, run `encore app init --lang ts` from the `apps/backend` directory. Then add the `prebuild` hook to the generated `encore.app` file: **apps/backend/encore.app:** ```json { "id": "generated-id", "lang": "typescript", "build": { "hooks": { "prebuild": "npx turbo build --filter=@repo/backend^..." } } } ``` The `prebuild` hook runs when deploying via Encore Cloud or when exporting a Docker image with the Encore CLI. The filter `@repo/backend^...` tells Turborepo to build all dependencies of `@repo/backend`. The `^` excludes the backend itself, building only its dependencies. **apps/backend/package.json:** ```json { "name": "@repo/backend", "version": "1.0.0", "type": "module", "scripts": { "dev": "encore run" }, "dependencies": { "@repo/shared": "*", "encore.dev": "latest" } } ``` ### Using the shared package With this setup, you can import from your shared package in your Encore services: **apps/backend/article/article.ts:** ```ts import { api } from "encore.dev/api"; import type { Article, CreateArticleRequest } from "@repo/shared"; import { slugify, truncate } from "@repo/shared"; export const create = api( { expose: true, method: "POST", path: "/article" }, async ({ title, content }: CreateArticleRequest): Promise

=> { return { slug: slugify(title), title: title, preview: truncate(content, 100), }; }, ); ``` ## Running the application ### Installation First, install all dependencies from the monorepo root: ```shell $ npm install ``` This installs dependencies for all workspaces, including Turborepo. ### Local development For local development, you need to build the shared packages before running `encore run`. From the monorepo root: ```shell $ npx turbo run build $ cd apps/backend && encore run ``` Or use Turborepo's `dev` task which handles the dependency ordering: ```shell $ npx turbo run dev --filter=@repo/backend ``` The `turbo.json` configuration ensures `@repo/shared` is built before the backend's dev task runs. ### Deployment When deploying via Encore Cloud or exporting a Docker image, the `prebuild` hook in `encore.app` automatically runs the Turborepo build pipeline. > **Note** > > When deploying a monorepo to Encore Cloud, configure the root path to your Encore app in the app settings: **Settings > General > Root Directory** (e.g., `apps/backend`). ## Key points - **Local development**: Run `npx turbo run build` before `encore run`, or use `npx turbo run dev --filter=@repo/backend` to handle dependency ordering automatically - **Prebuild hook**: The `prebuild` hook in `encore.app` runs during deployment (Encore Cloud) or Docker export, not during local development - **Turborepo filter**: Using `--filter=@repo/backend^...` builds only the dependencies of the backend (the `^` excludes the package itself) --- Source: https://encore.dev/docs/ts/develop/monorepo/nx Section: Building (Encore.ts) # Nx > Using Encore in an Nx monorepo [Nx](https://nx.dev) is a build system for JavaScript and TypeScript monorepos. This guide shows how to set up an Encore application within an Nx monorepo that depends on shared packages requiring compilation. ## Overview When using Encore in an Nx monorepo, you may have shared packages (like utility libraries or shared types) that need to be built before the Encore app can use them. Since Encore parses your application on startup, these dependencies must be compiled first. This guide covers two scenarios: - **Local development**: Use Nx to build dependencies before running `encore run` - **Deployment**: Use Encore's `prebuild` hook to automatically build dependencies when deploying via Encore Cloud or exporting a Docker image ## Project structure A typical Nx setup with Encore looks like this: ``` my-nx-workspace/ ├── apps/ │ └── backend/ # Encore application │ ├── encore.app │ ├── package.json │ ├── project.json │ ├── tsconfig.json │ └── article/ │ └── article.ts ├── packages/ │ └── shared/ # Shared library requiring build │ ├── package.json │ ├── project.json │ ├── tsconfig.json │ ├── src/ │ │ └── index.ts │ └── dist/ # Built output │ └── index.js ├── nx.json ├── package.json └── package-lock.json ``` ## Configuration ### Root package.json Configure npm workspaces to include your apps and packages: ```json -- package.json -- { "name": "my-nx-workspace", "private": true, "scripts": { "build": "nx run-many -t build", "dev": "nx run-many -t dev" }, "devDependencies": { "nx": "^21.0.0", "typescript": "^5.0.0" }, "workspaces": [ "apps/*", "packages/*" ] } ``` ### nx.json Configure Nx's build pipeline in the root `nx.json`: ```json -- nx.json -- { "$schema": "./node_modules/nx/schemas/nx-schema.json", "targetDefaults": { "build": { "dependsOn": ["^build"], "outputs": ["{projectRoot}/dist/**"], "cache": true }, "dev": { "cache": false } } } ``` The `"dependsOn": ["^build"]` configuration ensures that a project's dependencies are built before the project itself. ### Shared package Your shared package needs to compile TypeScript to JavaScript and expose the built output. **packages/shared/package.json:** ```json { "name": "@repo/shared", "version": "1.0.0", "private": true, "type": "module", "main": "./dist/index.js", "types": "./dist/index.d.ts", "exports": { ".": { "types": "./dist/index.d.ts", "default": "./dist/index.js" } }, "scripts": { "build": "tsc" }, "devDependencies": { "typescript": "^5.0.0" } } ``` **packages/shared/project.json:** ```json { "name": "@repo/shared", "$schema": "../../node_modules/nx/schemas/project-schema.json", "sourceRoot": "packages/shared/src", "projectType": "library", "targets": { "build": { "executor": "nx:run-commands", "options": { "command": "tsc", "cwd": "packages/shared" }, "outputs": ["{projectRoot}/dist"] } } } ``` **packages/shared/tsconfig.json:** ```json { "compilerOptions": { "outDir": "dist", "rootDir": "src", "moduleResolution": "bundler", "module": "ES2022", "target": "ES2022", "declaration": true }, "include": ["src"], "exclude": ["node_modules", "dist"] } ``` **packages/shared/src/index.ts:** ```ts // Types shared between frontend and backend export interface Article { slug: string; title: string; preview: string; } export interface CreateArticleRequest { title: string; content: string; } // Utility functions export function slugify(text: string): string { return text .toLowerCase() .trim() .replace(/[^\w\s-]/g, "") .replace(/\s+/g, "-"); } export function truncate(text: string, maxLength: number): string { if (text.length <= maxLength) return text; return text.slice(0, maxLength - 3) + "..."; } ``` ### Encore application The Encore app needs three key configurations: 1. **encore.app** - Use the `prebuild` hook to build dependencies during deployment 2. **package.json** - Declare the dependency on the shared package 3. **project.json** - Configure Nx targets and task dependencies To create the Encore app, run `encore app init --lang ts` from the `apps/backend` directory. Then add the `prebuild` hook to the generated `encore.app` file: **apps/backend/encore.app:** ```json { "id": "generated-id", "lang": "typescript", "build": { "hooks": { "prebuild": "npx nx build-deps @repo/backend" } } } ``` The `prebuild` hook runs when deploying via Encore Cloud or when exporting a Docker image with the Encore CLI. The `build-deps` target builds all dependencies of the backend. **apps/backend/package.json:** ```json { "name": "@repo/backend", "version": "1.0.0", "type": "module", "scripts": { "dev": "encore run" }, "dependencies": { "@repo/shared": "*", "encore.dev": "latest" } } ``` **apps/backend/project.json:** ```json { "name": "@repo/backend", "$schema": "../../node_modules/nx/schemas/project-schema.json", "sourceRoot": "apps/backend", "projectType": "application", "targets": { "dev": { "executor": "nx:run-commands", "options": { "command": "encore run", "cwd": "apps/backend" }, "dependsOn": ["^build"], "cache": false }, "build-deps": { "dependsOn": ["^build"], "cache": true } } } ``` The `"dependsOn": ["^build"]` configuration uses the `^` prefix to indicate "run the build target on all dependencies first". This automatically builds all shared packages before running `encore run`, without needing to list each dependency explicitly. ### Using the shared package With this setup, you can import from your shared package in your Encore services: **apps/backend/article/article.ts:** ```ts import { api } from "encore.dev/api"; import type { Article, CreateArticleRequest } from "@repo/shared"; import { slugify, truncate } from "@repo/shared"; export const create = api( { expose: true, method: "POST", path: "/article" }, async ({ title, content }: CreateArticleRequest): Promise
=> { return { slug: slugify(title), title: title, preview: truncate(content, 100), }; }, ); ``` ## Running the application ### Installation First, install all dependencies from the monorepo root: ```shell $ npm install ``` This installs dependencies for all workspaces, including Nx. ### Local development For local development, you need to build the shared packages before running `encore run`. From the monorepo root: ```shell $ npx nx build-deps @repo/backend $ cd apps/backend && encore run ``` Or use Nx's `dev` target which handles the dependency ordering: ```shell $ npx nx dev @repo/backend ``` The `"dependsOn": ["^build"]` configuration ensures all dependencies are built before the backend's dev target runs. ### Deployment When deploying via Encore Cloud or exporting a Docker image, the `prebuild` hook in `encore.app` automatically runs the Nx build. > **Note** > > When deploying a monorepo to Encore Cloud, configure the root path to your Encore app in the app settings: **Settings > General > Root Directory** (e.g., `apps/backend`). ## Key points - **Local development**: Run `npx nx build-deps @repo/backend` before `encore run`, or use `npx nx dev @repo/backend` to handle dependency ordering automatically - **Prebuild hook**: The `prebuild` hook in `encore.app` runs during deployment (Encore Cloud) or Docker export, not during local development - **Task dependencies**: Use `"dependsOn": ["^build"]` in `project.json` to automatically build all dependencies before running a target --- Source: https://encore.dev/docs/go Section: Building (Encore.go) # Encore.go > An open source infrastructure SDK for Go Encore.go is an open source infrastructure SDK that lets you declare the cloud infrastructure your application needs, such as databases, Pub/Sub topics, object storage, caches, cron jobs, and secrets, directly in your Go code. Instead of stitching together Terraform, YAML, and cloud consoles to wire up these resources, you declare them as typed values and use them through their SDK methods. Encore then takes care of running the matching infrastructure locally during development, and (with [the Encore platform](https://encore.dev/docs/platform/introduction)) provisioning the real cloud resources in your own AWS or GCP account when you deploy. **New to Encore?** The Quickstart guide builds and runs a Go application with two services, a database, and Pub/Sub in a few minutes. - [Start the Quickstart guide](https://encore.dev/docs/quick-start) ## What you get with Encore.go - **Infrastructure as Go**: Declare [SQL databases](https://encore.dev/docs/go/primitives/databases), [Pub/Sub topics and subscriptions](https://encore.dev/docs/go/primitives/pubsub), [object storage buckets](https://encore.dev/docs/go/primitives/object-storage), [caches](https://encore.dev/docs/go/primitives/caching), [cron jobs](https://encore.dev/docs/go/primitives/cron-jobs), and [secrets](https://encore.dev/docs/go/primitives/secrets) as values in your code. Encore understands how these resources are used and generates the runtime wiring for you. - **Local infrastructure, automatically**: Run `encore run` and Encore boots a local environment that mirrors production: real Postgres, a local Pub/Sub broker, local object storage, and so on. No Docker Compose files to maintain. - **Type-safe service-to-service calls**: Define APIs as plain Go functions. Calling another service is a normal function call; Encore handles serialization, routing, and validation at the boundary. - **Built-in observability**: A [local development dashboard](https://encore.dev/docs/dev-dash) with distributed [tracing](https://encore.dev/docs/tracing), logs, and a database explorer, so you can see what your application is doing without bolting on extra tools. - **AI-ready by design**: Because infrastructure is declared in code, AI coding assistants can understand and modify your full stack. Encore also ships [AI instructions](https://encore.dev/docs/ai-integration) and an [MCP server](https://encore.dev/docs/mcp) for editor integrations. ## The infrastructure primitives Each primitive is a package-level variable you declare in the same file as the code that uses it. Together they cover the infrastructure most backend applications run on. Anything outside the set connects as an external dependency. **SQL Database** ```go import "encore.dev/storage/sqldb" var db = sqldb.NewDatabase("orders", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` **Pub/Sub** ```go import "encore.dev/pubsub" var OrderEvents = pubsub.NewTopic[*OrderEvent]("order-events", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) ``` **Object Storage** ```go import "encore.dev/storage/objects" var Invoices = objects.NewBucket("invoices", objects.BucketConfig{}) ``` **Caching** ```go import "encore.dev/storage/cache" var OrdersCache = cache.NewCluster("orders-cache", cache.ClusterConfig{ EvictionPolicy: cache.AllKeysLRU, }) ``` **Cron Jobs** ```go import "encore.dev/cron" var _ = cron.NewJob("send-invoices", cron.JobConfig{ Title: "Send pending invoices", Every: 2 * cron.Hour, Endpoint: SendInvoices, }) ``` **Secrets** ```go var secrets struct { StripeAPIKey string // API key for the payments provider } ``` Declarations describe the infrastructure the application needs, and services and APIs are declared the same way. Provisioning settings are configured separately. Encore reads all of it into the [application model](https://encore.dev/docs/understanding-encore), which drives provisioning, IAM and code generation. Declarations are environment-independent, so the same application code runs in local, preview, and production environments without environment-specific checks. See [Development workflow](https://encore.dev/docs/platform/workflow) for the process from `encore run` through per-PR preview environments and production. ## Local and cloud implementations `encore run` starts a local implementation of each one, and in cloud environments [the Encore platform](https://encore.dev/docs/platform/introduction) provisions the managed equivalent in your own AWS or GCP account. You can also provision the infrastructure yourself and point Encore at it with an [infra config file](https://encore.dev/docs/self-host/configure-infra). | Primitive | Locally | AWS | GCP | |---|---|---|---| | [SQL Database](https://encore.dev/docs/go/primitives/databases) | Postgres in Docker | RDS | Cloud SQL | | [Pub/Sub](https://encore.dev/docs/go/primitives/pubsub) | in-memory | SNS + SQS | Cloud Pub/Sub | | [Object Storage](https://encore.dev/docs/go/primitives/object-storage) | local filesystem | S3 | Cloud Storage | | [Caching](https://encore.dev/docs/go/primitives/caching) | in-memory Redis | ElastiCache | Memorystore | | [Cron Jobs](https://encore.dev/docs/go/primitives/cron-jobs) | not triggered | Encore platform Managed | Encore platform Managed | | [Secrets](https://encore.dev/docs/go/primitives/secrets) | set per environment | Secrets Manager | Secret Manager | Cron jobs are not triggered automatically in local or preview environments. You can invoke their endpoints from the [development dashboard](https://encore.dev/docs/dev-dash). Compute is chosen per environment rather than declared in code: Cloud Run or GKE on GCP, Fargate or EKS on AWS. [Infrastructure on AWS and GCP](https://encore.dev/docs/platform/infrastructure/infra) also covers networking. ## Services and APIs Services and APIs are included in the application model, where Encore validates their definitions and relationships. - **[App Structure](https://encore.dev/docs/app-structure)**: how an application is laid out, and how services fit together in a monorepo. - **[Services](https://encore.dev/docs/go/primitives/services)**: group related APIs and infrastructure into independently deployable services. - **[Defining APIs](https://encore.dev/docs/go/primitives/defining-apis)**: expose typed endpoints from a service. Encore handles request validation, routing and client generation. - **[API Calls](https://encore.dev/docs/go/primitives/api-calls)**: call another service's API as a regular typed function, wired up in-process locally and over the network in production. ### Request and response data - **[Validation](https://encore.dev/docs/go/develop/validation)**: incoming requests are checked against the declared schema before your handler runs. - **[API Errors](https://encore.dev/docs/go/primitives/api-errors)**: return structured errors with codes that map to HTTP statuses. ### Other API styles - **[Raw Endpoints](https://encore.dev/docs/go/primitives/raw-endpoints)**: access the underlying request and response objects. - **[Service Structs](https://encore.dev/docs/go/primitives/service-structs)**: define APIs as methods on a struct, with dependencies initialized once per service. ## From local to production Because the SDK is the source of truth for your infrastructure, the same model runs locally, in per-PR preview environments, and in production. `encore run` boots the whole system on your laptop with real Postgres, real Pub/Sub semantics, and real tracing. Opening a pull request spins up a [preview environment](https://encore.dev/docs/platform/deploy/preview-environments) with the same infrastructure model and (optionally) a [database branched from a seed environment](https://encore.dev/docs/platform/infrastructure/neon). This is the loop that makes Encore.go particularly effective with AI coding agents: every change can be validated end-to-end against real infrastructure, locally and in the cloud, instead of waiting for a production deploy to find out if it works. For more, see the [Development Workflow](https://encore.dev/docs/platform/workflow) page. Encore.go is fully open source and you are never locked into a particular deployment path. There are two supported ways to run an Encore.go application in production. **With [the Encore platform](https://encore.dev/docs/platform/introduction)**: the optional managed platform reads the infrastructure you declared in your code and provisions matching resources (RDS, Cloud SQL, SNS+SQS, Pub/Sub, S3, GCS, etc.) in your own AWS or GCP account, then manages deploys, environments, secrets, and IAM on top. You own the cloud account and the infrastructure; the Encore platform is the control plane that drives it from your code. **Self-hosted**: build a standard Docker image with `encore build docker` and run it anywhere. You provision your own infrastructure however you like (Terraform, Pulumi, CloudFormation, the cloud console), and tell the Encore runtime how to reach those resources via an [infrastructure config file](https://encore.dev/docs/self-host/configure-infra). See the [self-hosting guide](https://encore.dev/docs/self-host/build) for the full workflow. ## Primitives and AI agents Primitives expose infrastructure resources as typed objects. Deployment-specific settings are configured per environment. This separation helps agents produce valid declarations, with invalid usage reported by the type system or at build time. For example, an agent can declare a database with only a name and a migrations directory: ```go var db = sqldb.NewDatabase("orders", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` An equivalent Terraform configuration may also define the database instance class, storage type, networking, parameter group, and IAM policy. Encore handles these concerns as follows: - Deployment settings such as instance size, networking, and backups are configured per [environment](https://encore.dev/docs/platform/infrastructure/configuration) and remain outside the application code. - Resource names must be string literals, and resource declarations must be defined at package scope. Invalid declarations are reported at build time. See [Understanding Encore](https://encore.dev/docs/understanding-encore) for the complete requirements. - IAM policies are derived from resource usage, limiting permissions to the relationships declared in the application. - Request schemas are derived from Go types, so invalid client usage and handler signatures produce compile-time errors. See [AI integration](https://encore.dev/docs/ai-integration) for editor rules and an MCP server that expose the service graph, schemas, and traces to agents. [Coming from Terraform](https://encore.dev/docs/platform/migration/from-terraform) maps Terraform concepts to Encore and covers using both together. ## Other infrastructure Primitives do not limit the infrastructure your application can use. For services without a primitive, such as a search cluster or data warehouse, provision them separately, connect to them as external dependencies, and store their connection details in [secrets](https://encore.dev/docs/go/primitives/secrets). Resources you already run can be connected to a declaration rather than provisioned new. Encore imports existing databases, buckets, and topics on [AWS](https://encore.dev/docs/platform/infrastructure/aws) and [GCP](https://encore.dev/docs/platform/infrastructure/gcp), and deploys into an existing [Kubernetes cluster](https://encore.dev/docs/platform/infrastructure/import-kubernetes-cluster). To see exactly what Encore creates in your cloud, see [Infrastructure on AWS and GCP](https://encore.dev/docs/platform/infrastructure/infra). ## Where to go next - New to Encore? Start with the [Quickstart guide](https://encore.dev/docs/quick-start) to build and run your first application. - Want to understand the model first? Read [App Structure](https://encore.dev/docs/app-structure) and the [Services](https://encore.dev/docs/go/primitives/services) page. - Curious about the local-to-production iteration loop? See the [Development Workflow](https://encore.dev/docs/platform/workflow). - Ready to deploy? See [Deploy with the Encore platform](https://encore.dev/docs/platform/deploy/deploying) or [Build Docker Images](https://encore.dev/docs/self-host/build) for self-hosting. ## Get started **Install the Encore CLI** (pick the line for your OS): ```shell # macOS brew install encoredev/tap/encore # Windows iwr https://encore.dev/install.ps1 | iex # Linux curl -L https://encore.dev/install.sh | bash ``` Then scaffold an app: ```shell encore app create ``` --- Source: https://encore.dev/docs/go/primitives/services Section: Building (Encore.go) # Defining Services > Simplifying (micro-)service development Encore.go makes it simple to build applications with one or many services, without needing to manually handle the typical complexity of developing microservices. ## Defining a service With Encore.go you define a service by [defining at least one API](https://encore.dev/docs/go/primitives/defining-apis) within a regular Go package. Encore recognizes this as a service, and uses the package name as the service name. On disk it might look like this: ``` /my-app ├── encore.app // ... and other top-level project files │ ├── hello // hello service (a Go package) │   ├── hello.go // hello service code │   └── hello_test.go // tests for hello service │ └── world // world service (a Go package) └── world.go // world service code ``` This means building a microservices architecture is as simple as creating multiple Go packages within your application. See the [app structure documentation](https://encore.dev/docs/app-structure) for more details. **Related example:** [encoredev/examples/trello-clone](https://github.com/encoredev/examples/tree/main/trello-clone) Simple microservices example application. Create it locally with `encore app create --example=trello-clone`. ## Service Initialization Under the hood Encore automatically generates a `main` function that initializes all your infrastructure resources when the application starts up. This means you don't write a `main` function for your Encore application. If you want to customize the initialization behavior of your service, you can define a service struct and define custom initialization logic with that. See the [service struct docs](https://encore.dev/docs/go/primitives/service-structs) for more info. --- Source: https://encore.dev/docs/go/how-to/atlas-gorm Section: Building (Encore.go) # Use Atlas + GORM for database migrations [Atlas](https://atlasgo.io) is a popular tool for managing database migrations. [GORM](https://gorm.io/) is a popular ORM for Go. Encore provides excellent support for using them together to easily manage database schemas and migrations. Encore executes database migrations using [golang-migrate](https://github.com/golang-migrate/migrate), which Atlas supports out-of-the-box. This means that you can use Atlas to manage your Encore database migrations. The easiest way to use Atlas + GORM together is with Atlas's support for [external schemas](https://atlasgo.io/blog/2023/06/28/external-schemas-and-gorm-support). ## Setting up GORM To set up your Encore application with GORM, start by installing the GORM package and associated Postgres driver: ```shell go get -u gorm.io/gorm gorm.io/driver/postgres ``` Then, in the service that you want to use GORM for, add the `*gorm.DB` as a dependency in your service struct (create a service struct if you don't already have one). For example, if you had a service called `blog`: ```go -- blog/blog.go -- package blog import ( "encore.dev/storage/sqldb" "gorm.io/driver/postgres" "gorm.io/gorm" ) //encore:service type Service struct { db *gorm.DB } var blogDB = sqldb.NewDatabase("blog", sqldb.DatabaseConfig{ Migrations: "./migrations", }) // initService initializes the site service. // It is automatically called by Encore on service startup. func initService() (*Service, error) { db, err := gorm.Open(postgres.New(postgres.Config{ Conn: blogDB.Stdlib(), })) if err != nil { return nil, err } return &Service{db: db}, nil } ``` Finally, create the `migrations` directory inside the `blog` directory if it doesn't already exist. This is where Atlas will put your database migrations. ## Setting up Atlas First [install Atlas](https://atlasgo.io/getting-started). Then, add an `atlas.hcl` file inside the `blog` directory: ``` -- blog/atlas.hcl -- data "external_schema" "gorm" { program = ["env", "ENCORERUNTIME_NOPANIC=1", "go", "run", "./scripts/atlas-gorm-loader.go"] } env "local" { src = data.external_schema.gorm.url migration { dir = "file://migrations" format = golang-migrate } format { migrate { diff = "{{ sql . \" \" }}" } } } ``` Next, we need to create the `atlas-gorm-loader` script referenced above. It will use the [atlas-provider-gorm](https://github.com/ariga/atlas-provider-gorm) library provided by Atlas. Create the file as follows: ``` -- blog/scripts/atlas-gorm-loader.go -- package main import ( "fmt" "io" "os" _ "ariga.io/atlas-go-sdk/recordriver" "ariga.io/atlas-provider-gorm/gormschema" "encore.app/blog" ) // Define the models to generate migrations for. var models = []any{ &blog.Post{}, &blog.Comment{}, } func main() { stmts, err := gormschema.New("postgres").Load(models...) if err != nil { fmt.Fprintf(os.Stderr, "failed to load gorm schema: %v\n", err) os.Exit(1) } io.WriteString(os.Stdout, stmts) } ``` ## Creating migrations To wrap things up, let's create a script to automate the process of generating migrations: ``` -- blog/scripts/generate-migration -- #!/bin/bash set -eu DB_NAME=blog MIGRATION_NAME=${1:-} SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) # Reset the shadow database encore db reset --shadow $DB_NAME # GORM executes Go code without initializing Encore when generating migrations, # so configure the Encore runtime to be aware that this is expected. export ENCORERUNTIME_NOPANIC=1 # Generate the migration atlas migrate diff $MIGRATION_NAME --env local --dev-url "$(encore db conn-uri --shadow $DB_NAME)&search_path=public" ``` Finally let's make the script executable, and generate our first migration: ```shell $ chmod +x blog/scripts/generate-migration $ cd blog && ./scripts/generate-migration init ``` This will generate a new migration file in the `blog/migrations` directory, which will be automatically applied when running `encore run`. --- Source: https://encore.dev/docs/go/how-to/entgo-orm Section: Building (Encore.go) # Use ent ORM + Atlas for database schemas Encore has all the tools needed to support ORMs and migration frameworks out-of-the-box through [named databases](https://encore.dev/docs/go/primitives/share-db-between-services) and [migration files](https://encore.dev/docs/go/primitives/databases#defining-the-database-schema). Writing plain SQL might not work for your use case, or you may not want to use SQL in the first place. ORMs like [ent](https://entgo.io/) and migration frameworks like [Atlas](https://atlasgo.io/) can be used with Encore by integrating their logic with a system's database. Encore is not restrictive, it uses plain SQL migration files for its migrations. - If your ORM of choice can connect to any database using a [standard SQL driver](https://github.com/lib/pq), then it can be used with Encore. - If your migration framework can generate SQL migration files without any modifications, then it can be used with Encore. Let's take a look at how you can integrate ent with Encore, using Atlas for generating the migration files. ## Add ent schemas to a service [Install ent](https://entgo.io/docs/tutorial-setup#installation), then initialize your first schema in the service where you want to use it. For example, if you had the following app structure: ``` /my-app ├── encore.app └── user // user service ``` You can then use this command to generate a user schema along with the ent directory that will contain that schema and all future generated files: ```shell $ go run entgo.io/ent/cmd/ent@latest new --target user/ent/schema User ``` The `--target` option sets the schema directory within your Encore system. Each system should contain its own models and schemas, and its own migration files. Like you would when using plain SQL. Add the fields and edges for your new model in the generated file under `user/ent/schema/user.go`. Now, run the following command: ```shell $ go run entgo.io/ent/cmd/ent@latest generate ./user/ent/schema ``` This generates the ent client files. Run this command again whenever you change the schemas. ## Integrating ent with an Encore database Encore automates database provisioning, and automatically runs migrations in all environments. To integrate ent with Encore, we need to do three things: 1. Create the Encore database 2. Set up the ent client to use that database. 3. Generate migration files for the ent schema, using Atlas. ### Create the Encore database Create the database using [`sqldb.NewDatabase`](https://encore.dev/docs/go/primitives/databases) in `user/user.go`: ``` -- user/user.go -- package user import "encore.dev/storage/sqldb" var userDB = sqldb.NewDatabase("user", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` Now, create the `migrations` directory, and leave it empty for now: ```shell $ mkdir user/migrations ``` ### Connect ent to the database Next, extend the user service with a [Service Struct](https://encore.dev/docs/go/primitives/service-structs) that creates an ent client connected to the database. Replace the contents of the `user/user.go` file with: ``` -- user/user.go -- package user import ( "encore.dev/storage/sqldb" "entgo.io/ent/dialect" entsql "entgo.io/ent/dialect/sql" "encore.app/user/ent" ) var userDB = sqldb.NewDatabase("user", sqldb.DatabaseConfig{ Migrations: "./migrations", }) //encore:service type Service struct{ ent *ent.Client } func initService() (*Service, error) { driver := entsql.OpenDB(dialect.Postgres, userDB.Stdlib()) entClient := ent.NewClient(ent.Driver(driver)) return &Service{ent: entClient}, nil } ``` Now ent is fully wired up to the Encore database, and can be used from the service struct in any API endpoint. ## Using Atlas for database migrations Finally, we'll set up Atlas to generate database migrations for the ent schema. First, make sure you [have Atlas installed](https://atlasgo.io/getting-started). Then, create the file `user/atlas.hcl` containing the following: ``` -- user/atlas.hcl -- env "local" { src = "ent://ent/schema" migration { dir = "file://migrations" format = golang-migrate } format { migrate { diff = "{{ sql . \" \" }}" } } } ``` This tells Atlas to generate migrations for the ent schema, and to output them to the `migrations` directory. Atlas works by comparing the desired ent schema with the current database schema, and generating a migration to bring the database schema in line with the ent schema. This relies on a so-called "shadow database", which is an empty database that Atlas uses to compare the ent schema against. Fortunately for us, Encore has built-in support for shadow databases. Create the file `user/scripts/generate-migration` containing the following: ``` -- user/scripts/generate-migration -- #!/bin/bash set -eu DB_NAME=user MIGRATION_NAME=${1:-} # Reset the shadow database encore db reset --shadow $DB_NAME # ent executes Go code without initializing Encore when generating migrations, # so configure the Encore runtime to be aware that this is expected. export ENCORERUNTIME_NOPANIC=1 # Generate the migration atlas migrate diff $MIGRATION_NAME --env local --dev-url "$(encore db conn-uri --shadow $DB_NAME)&search_path=public" ``` Finally, make the script executable, and generate our first migration: ```shell $ chmod +x user/scripts/generate-migration $ cd user && ./scripts/generate-migration init ``` You should see a new migration file being added to the `user/migrations` directory, containing the schema changes to create the ent models. You can now run the service with `encore run`, and everything should be ready to go! --- Source: https://encore.dev/docs/go/primitives/change-db-schema Section: Building (Encore.go) # Change SQL database schema Encore database schemas are changed over time using *migration files*. Each migration file has a sequence number, and migration files are run in sequence when deploying. Encore tracks which migrations have already run and only runs new ones. To change your database schema, add a new migration file using the next available migration number. For example, if you have two migration files already, the next migration file should be named `3_something.up.sql` where `something` is a short description of what the migration does. > **Warning** > > Database migrations are applied before the application is restarted > with the new code. Always make sure the old application code works with > the new database schema, so that things don't break while your new code > is being rolled out. ## Example Let's say you have a single migration file that creates a `todo_item` table: **`todo/migrations/1_create_table.up.sql`** ```sql CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL ); ``` And now you want to add a `created` column to track when each todo was created. Add a new file: **`todo/migrations/2_add_created_col.up.sql`** ```sql ALTER TABLE todo_item ADD created TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(); ``` The next deploy Encore will notice the new migration file and run it, adding a new column. --- Source: https://encore.dev/docs/go/primitives/connect-existing-db Section: Building (Encore.go) # Integrate with existing databases Encore automatically provision the necessary infrastructure when you create a service and add a database. However, you may want to connect to an existing database for migration or prototyping purposes. It's simple to integrate your Encore app with an existing database in these cases. ## Example Let's say you have an external database hosted by DigitalOcean that you would like to connect to. The simplest approach is to create a dedicated package that lazily instantiates a database connection pool. We can store the password using Encore's [secrets manager](https://encore.dev/docs/go/primitives/secrets) to make it even easier. The connection string is something that looks like: ``` postgresql://user:password@externaldb-do-user-1234567-0.db.ondigitalocean.com:25010/externaldb?sslmode=require ``` So we write something like: **`pkg/externaldb/externaldb.go`** ```go package externaldb import ( "context" "fmt" "github.com/jackc/pgx/v4/pgxpool" "go4.org/syncutil" ) // Get returns a database connection pool to the external database. // It is lazily created on first use. func Get(ctx context.Context) (*pgxpool.Pool, error) { // Attempt to setup the database connection pool if it hasn't // already been successfully setup. err := once.Do(func() error { var err error pool, err = setup(ctx) return err }) return pool, err } var ( // once is like sync.Once except it re-arms itself on failure once syncutil.Once // pool is the successfully created database connection pool, // or nil when no such pool has been setup yet. pool *pgxpool.Pool ) var secrets struct { // ExternalDBPassword is the database password for authenticating // with the external database hosted on DigitalOcean. ExternalDBPassword string } // setup attempts to set up a database connection pool. func setup(ctx context.Context) (*pgxpool.Pool, error) { connString := fmt.Sprintf("postgresql://%s:%s@externaldb-do-user-1234567-0.db.ondigitalocean.com:25010/externaldb?sslmode=require", "user", secrets.ExternalDBPassword) return pgxpool.Connect(ctx, connString) } ``` Before running, remember to use `encore secrets set` to store the `ExternalDBPassword` to use. (But don't worry, Encore will remind you if you forget.) ## Other infrastructure The same pattern can easily be adapted to other infrastructure components that Encore doesn't yet provide built-in support for: - Horizontally scalable databases like Cassandra, DynamoDB, BigTable, and so on - Document or graph databases like MongoDB or Neo4j - Other cloud primitives like queues, object storage buckets, and more - Or really any cloud services or APIs you can think of In this way you can easily integrate Encore with anything you want. --- Source: https://encore.dev/docs/go/primitives/insert-test-data-db Section: Building (Encore.go) # Insert test data in a database When you're developing or testing, it's often useful to seed databases with test data. This can be done is several ways depending on your use case. ## Using go:embed A straightforward way to insert test data is to conditionally insert it on startup using `go:embed` in combination with Encore's [metadata API](https://encore.dev/docs/go/develop/metadata) control in which environments the data gets inserted. E.g. only in your local environment. ### Example Create a file with your test data named `fixtures.sql`. Then, for the service where you want to insert test data, add the following to its `.go` file in order to run on startup, where `db` is the service's database declared with `sqldb.NewDatabase`. ``` import ( "context" _ "embed" "log" "encore.dev" ) //go:embed fixtures.sql var fixtures string func init() { if encore.Meta().Environment.Cloud == encore.CloudLocal { if _, err := db.Exec(context.Background(), fixtures); err != nil { log.Fatalln("unable to add fixtures:", err) } } } ``` Not included in the above example is preventing adding duplicate data. This is straightforward to do by making the fixtures idempotent, or by tracking it with a database table. ## Populating databases in Encore Cloud Preview Environments If you are using Encore Cloud Preview Environments, it can sometimes be useful to populate new Preview Environments with test data to simplify testing. The best way to do this depends a bit on your use case, but a common way to do this is by using Encore's [webhooks](https://encore.dev/docs/platform/integrations/webhooks) functionality, which provides notifications for when a deployment is completed and includes information about the environment in question. --- Source: https://encore.dev/docs/go/primitives/share-db-between-services Section: Building (Encore.go) # Share SQL databases between services By default, each service in an Encore app has its own database. This approach has many benefits: - Which database is used and how it works is abstracted away from other services - The database is more isolated, making changes to it smaller and safer - By making the services more independent your application becomes more reliable by being able to more gracefully handle partial outages, such as if your database is temporarily overloaded or offline. But like everything else in software engineering, there are trade-offs involved, and sometimes it's simpler and more reliable to use a single database that's accessed by multiple services. Encore makes this easy to do. Each database in Encore is defined within a service. That service's name becomes the name of the database. Other services can then access that database by creating a database reference with `sqldb.Named("dbname")`. ## Example Let's say you have a simple `todo` service, with only one table: **`todo/migrations/1_create_table.up.sql`** ```sql CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL DEFAULT FALSE ); ``` You want to create a `report` service that produces various reports for internal business processes, but for simplicity you decide it makes sense to directly access the `todo` database. All that's needed is to define the `todoDB` variable like so: **`report/report.go`** ```go package report import ( "context" "encore.dev/storage/sqldb" ) // todoDB connects to the "todo" service's database. var todoDB = sqldb.Named("todo") type ReportResponse struct { Total int } // CountCompletedTodos generates a report with the number of completed todo items. //encore:api method=GET path=/report/todo func CountCompletedTodos(ctx context.Context) (*ReportResponse, error) { var report ReportResponse err := todoDB.QueryRow(ctx,` SELECT COUNT(*) FROM todo_item WHERE completed = TRUE `).Scan(&report.Total) return &report, err } ``` With that, Encore understands that the `report` service depends on the `todo` service's database, and orchestrates the necessary connections to make that happen. And like everything else with Encore, it works exactly the same regardless of where it's running: for local development as well as in the cloud. --- Source: https://encore.dev/docs/go/primitives/databases/troubleshooting Section: Building (Encore.go) # Troubleshooting Databases > Advice on troubleshooting SQL databases in Encore.go When you run your application locally with `encore run`, Encore provisions local databases using [Docker](https://docker.com). If this fails with a database error, it can often be resolved by making sure you have Docker installed and running, or by restarting the Encore daemon using `encore daemon`. If this does not resolve the issue, here are steps to resolve common errors: ** Error: sqldb: unknown database ** This error is often caused by a problem with the initial migration file, such as incorrect naming or location. - Verify that you've [created the migration file](https://encore.dev/docs/go/primitives/databases#defining-the-database-schema) correctly, then try `encore run` again. ** Error: could not connect to the database ** When you can't connect to the database in your local environment, there's likely an issue with Docker: - Make sure that you have [Docker](https://docker.com) installed and running, then try `encore run` again. - If this fails, restart the Encore daemon by running `encore daemon`, then try `encore run` again. ** Error: Creating PostgreSQL database cluster Failed ** This means Encore was not able to create the database. Often this is due to a problem with Docker. - Check if you have permission to access Docker by running `docker images`. - Set the correct permissions with `sudo usermod -aG docker $USER` (Learn more in the [Docker documentation](https://docs.docker.com/engine/install/linux-postinstall/)) - Then log out and log back in so that your group membership is refreshed. ** Error: unable to save docker image ** This error is often caused by a problem with Docker. - Make sure that you have [Docker](https://docker.com) installed and running. - In Docker, open **Settings > Advanced** and make sure that the setting `Allow the default Docker socket to be used` is checked. - If it still fails, restart the Encore daemon by running `encore daemon`, then try `encore run` again. ** Error: unable to add CA to cert pool ** This error is commonly caused by the presence of the file `$HOME/.postgresql/root.crt` on the filesystem. When this file is present the PostgreSQL client library will assume the database server has that root certificate, which will cause the above error. - Remove or rename the file, then try `encore run` again. ** Resetting databases ** If your local database is in a bad state (e.g. due to a incomplete migration or corrupt data), you can reset it by running: ```shell $ encore db reset ``` This drops and recreates the database, re-running all migrations from scratch. Use `--all` to reset all databases at once. --- Source: https://encore.dev/docs/go/primitives/databases Section: Building (Encore.go) # Using SQL databases > Provisioning, migrating, querying Encore treats SQL databases as logical resources and natively supports **PostgreSQL** databases. ## Creating a database To create a database, import `encore.dev/storage/sqldb` and call `sqldb.NewDatabase`, assigning the result to a package-level variable. Databases must be created from within an [Encore service](https://encore.dev/docs/go/primitives/services). For example: ``` -- todo/db.go -- package todo // Create the todo database and assign it to the "tododb" variable var tododb = sqldb.NewDatabase("todo", sqldb.DatabaseConfig{ Migrations: "./migrations", }) // Then, query the database using db.QueryRow, db.Exec, etc. -- todo/migrations/1_create_table.up.sql -- CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL DEFAULT false -- etc... ); ``` As seen above, the `sqldb.DatabaseConfig` specifies the directory containing the database migration files, which is how you define the database schema. See the [Defining the database schema](#defining-the-database-schema) section below for more details. With this code in place, Encore will automatically create the database using [Docker](https://docker.com) when you run the command `encore run` in your local environment. Make sure Docker is installed and running on your machine before running `encore run`. > **Note** > > If your application is already running when you define a new database, you will need to stop and restart `encore run`. This is necessary for Encore to create the new database using Docker. **Related example:** [encoredev/examples/sql-database](https://github.com/encoredev/examples/tree/main/sql-database) Simple PostgreSQL example application. Create it locally with `encore app create --example=sql-database`. ## Database Migrations Encore automatically handles `up` migrations, while `down` migrations must be run manually. Each `up` migration runs sequentially, expressing changes in the database schema from the previous migration. ### Naming Conventions **File Name Format:** Migration files must start with a number followed by an underscore (`_`), and must increase sequentially. Each file name must end with `.up.sql`. **Examples:** - `1_first_migration.up.sql` - `2_second_migration.up.sql` - `3_migration_name.up.sql` You can also prefix migration files with leading zeroes for better ordering in the editor (e.g., `0001_migration.up.sql`). ### Defining the Database Schema The first migration typically defines the initial table structure. For instance, a `todo` service might create `todo/migrations/1_create_table.up.sql` with the following content: ```sql -- todo/migrations/1_create_table.up.sql -- CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL DEFAULT false ); ``` ### Migration File Structure Migration files are created in a `migrations` directory within an Encore service package. Each file is named `_.up.sql`, where `` is a sequence number for ordering and `` describes the migration. **Example Directory Structure:** ``` /my-app ├── encore.app // ... and other top-level project files │ └── todo // todo service (a Go package)    ├── migrations // todo service db migrations (directory)    │ ├── 1_create_table.up.sql // todo service db migration    │ └── 2_add_field.up.sql // todo service db migration    ├── todo.go // todo service code    └── todo_test.go // tests for todo service ``` ## Inserting data into databases Once you have created the database using `var mydb = sqldb.NewDatabase(...)` you can start inserting data into the database by calling methods on the `mydb` variable. The interface is similar to that of the Go standard library's `database/sql` package. Learn more in the [package docs](https://pkg.go.dev/encore.dev/storage/sqldb). One way of inserting data is with a helper function that calls `Exec` on the database. For example, to insert a single todo item using the example schema above, we can use the following helper function `insert`: ``` -- todo/insert.go -- // insert inserts a todo item into the database. func insert(ctx context.Context, id, title string, done bool) error { _, err := tododb.Exec(ctx, ` INSERT INTO todo_item (id, title, done) VALUES ($1, $2, $3) `, id, title, done) return err } -- todo/db.go -- package todo // Create the todo database and assign it to the "tododb" variable var tododb = sqldb.NewDatabase("todo", sqldb.DatabaseConfig{ Migrations: "./migrations", }) // Then, query the database using db.QueryRow, db.Exec, etc. -- todo/migrations/1_create_table.up.sql -- CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL DEFAULT false -- etc... ); ``` ## Querying databases To query a database in your application, you similarly need to import `encore.dev/storage/sqldb` in your service package or sub-package. For example, to read a single todo item in the example schema above, we can use `sqldb.QueryRow`: ```go var item struct { ID int64 Title string Done bool } err := tododb.QueryRow(ctx, ` SELECT id, title, done FROM todo_item LIMIT 1 `).Scan(&item.ID, &item.Title, &item.Done) ``` If `QueryRow` does not find a matching row, it reports an error that can be checked against by importing the standard library `errors` package and calling `errors.Is(err, sqldb.ErrNoRows)`. Learn more in the [package docs](https://pkg.go.dev/encore.dev/storage/sqldb). ## Provisioning databases Encore automatically provisions databases to match what your application requires. When you [define a database](#creating-a-database), Encore will provision the database at your next deployment. Encore provisions databases in an appropriate way depending on the environment. When running locally, Encore creates a database cluster using [Docker](https://www.docker.com/). In the cloud, it depends on the [environment type](https://encore.dev/docs/platform/deploy/environments#environment-types): - In `production` environments, the database is provisioned through the Managed SQL Database service offered by the chosen cloud provider. - In `development` environments, the database is provisioned as a Kubernetes deployment with a persistent disk attached. See exactly what is provisioned for each cloud provider, and each environment type, in the [infrastructure documentation](https://encore.dev/docs/platform/infrastructure/infra). ## Connecting to databases It's often useful to be able to connect to the database from outside the backend application. For example for scripts, ad-hoc querying, or dumping data for analysis. Currently Encore does not expose user credentials for databases in the local environment or for environments on Encore Cloud. You can use a connection string to connect instead, see below. If you need to connect external tools like data pipelines or BI platforms, see [Connecting external tools to databases](https://encore.dev/docs/platform/infrastructure/managing-infrastructure#connecting-external-tools-to-databases) for guidance on SSL certificates and network access. ### Using the Encore CLI Encore's CLI comes with built-in support for connecting to databases: * `encore db shell [--env=]` opens a [psql](https://www.postgresql.org/docs/current/app-psql.html) shell to the database named `` in the given environment. Leaving out `--env` defaults to the local development environment. `encore db shell` defaults to read-only permissions. Use `--write`, `--admin` and `--superuser` flags to modify which permissions you connect with. * `encore db conn-uri [--env=]` outputs a connection string for the database named ``. When specifying a cloud environment, the connection string is temporary. Leaving out `--env` defaults to the local development environment. * `encore db proxy [--env=]` sets up a local proxy that forwards any incoming connection to the databases in the specified environment. Leaving out `--env` defaults to the local development environment. See `encore help db` for more information on database management commands. ### Using database user credentials For cloud environments on AWS/GCP you can view database user credentials (created by Encore when provisioning databases) via the Encore platform dashboard: * Open your app in the [Encore platform dashboard](https://app.encore.dev), navigate to the **Infrastructure** page for the appropriate environment, and locate the `USERS` section within the relevant **Database Cluster**. ## Handling migration errors When Encore applies database migrations, there's always a possibility the migrations don't apply cleanly. This can happen for many reasons: - There's a problem with the SQL syntax in the migration - You tried to add a `UNIQUE` constraint but the values in the table aren't actually unique - The existing database schema didn't look like you thought it did, so the database object you tried to change doesn't actually exist - ... and so on If that happens, Encore rolls back the migration. If it happens during a cloud deployment, the deployment is aborted. Once you fix the problem, re-run `encore run` (locally) or push the updated code (in the cloud) to try again. Encore tracks which migrations have been applied in the `schema_migrations` table: ```sql database=# \d schema_migrations Table "public.schema_migrations" Column | Type | Collation | Nullable | Default ---------+---------+-----------+----------+--------- version | bigint | | not null | dirty | boolean | | not null | Indexes: "schema_migrations_pkey" PRIMARY KEY, btree (version) ``` The `version` column tracks which migration was last applied. If you wish to skip a migration or re-run a migration, change the value in this column. For example, to re-run the last migration, run `UPDATE schema_migrations SET version = version - 1;`. *Note that Encore does not use the `dirty` flag by default.* ## The `encore_services` role Encore uses a shared database role named `encore_services` to bridge permissions between database migrations and the services that connect to the database at runtime. This role exists consistently across all environments — local and cloud. - All roles that run migrations are granted `encore_services`. - All roles used by services to connect to the database at runtime are also granted `encore_services`. This means anything owned by `encore_services` — or any permissions granted to it — is automatically accessible to both migration roles and runtime service roles. You can use migration scripts to grant additional permissions to `encore_services`, and they will be inherited by the service roles connecting at runtime. ### Materialized views example Materialized views are a good example of where `encore_services` is useful: they're created during migrations, but need to be refreshed at runtime by the service role. You can bridge this gap by creating a dedicated owner role in a migration, granting it to `encore_services`, and assuming it when creating the view: ```sql -- In a migration file CREATE ROLE matview_owner; GRANT matview_owner TO encore_services; SET ROLE matview_owner; CREATE MATERIALIZED VIEW my_view AS SELECT id, count(*) AS total FROM todo_item GROUP BY id; RESET ROLE; ``` At runtime, the service role can assume `matview_owner` (via `encore_services`) and refresh the view: ```go _, err := tododb.Exec(ctx, `SET ROLE matview_owner`) _, err = tododb.Exec(ctx, `REFRESH MATERIALIZED VIEW my_view`) _, err = tododb.Exec(ctx, `RESET ROLE`) ``` ### Legacy behavior In earlier versions of Encore, migrations were run using the same database account that services used to connect at runtime. To restore this legacy behavior, set the following environment variable: ```shell ENCOREDEBUG="sqldbrole=legacy" ``` This is provided for backwards compatibility and is not recommended for new applications. --- Source: https://encore.dev/docs/go/primitives/object-storage Section: Building (Encore.go) # Object Storage > Simple and scalable storage APIs for files and unstructured data Object Storage is a simple and scalable solution to store files and unstructured data in your backend application. The most common implementation is Amazon S3 ("Simple Storage Service") and its semantics are universally supported by every major cloud provider. Encore.go provides a cloud-agnostic API for working with Object Storage, allowing you to store and retrieve files with ease. It has support for Amazon S3, Google Cloud Storage, as well as any other S3-compatible implementation (such as DigitalOcean Spaces, MinIO, etc.). Additionally, when you use Encore's Object Storage API you also automatically get: * Automatic tracing and instrumentation of all Object Storage operations * Built-in local development support, storing objects on the local filesystem * Support for integration testing, using a local, in-memory storage backend ## Creating a Bucket The core of Object Storage is the **Bucket**, which represents a collection of files. In Encore, buckets must be declared as package level variables, and cannot be created inside functions. Regardless of where you create a bucket, it can be accessed from any service by referencing the variable it's assigned to. When creating a bucket you can configure additional properties, like whether the objects in the bucket should be versioned. See the complete specification in the [package documentation](https://pkg.go.dev/encore.dev/storage/objects#NewBucket). For example, to create a bucket for storing profile pictures: ```go package user import "encore.dev/storage/objects" var ProfilePictures = objects.NewBucket("profile-pictures", objects.BucketConfig{ Versioned: false, }) ``` ## Uploading files To upload a file to a bucket, use the `Upload` method on the bucket variable. It returns a writer that you can use to write the contents of the file. To complete the upload, call the `Close` method on the writer. To abort the upload, either cancel the context or call the `Abort` method on the writer. The `Upload` method additionally takes a set of options to configure the upload, like setting attributes (`objects.WithUploadAttrs`) or to reject the upload if the object already exists (`objects.WithPreconditions`). See the [package documentation](https://pkg.go.dev/encore.dev/storage/objects#Bucket.Upload) for more details. ```go package user import ( "context" "io" "net/http" "encore.dev/beta/auth" "encore.dev/beta/errs" "encore.dev/storage/objects" ) var ProfilePictures = objects.NewBucket("profile-pictures", objects.BucketConfig{}) //encore:api auth raw method=POST path=/upload-profile-picture func UploadProfilePicture(w http.ResponseWriter, req *http.Request) { // Store the user's profile picture with their user id as the key. userID, _ := auth.UserID() key := string(userID) // We store the profile writer := ProfilePictures.Upload(req.Context(), key) _, err := io.Copy(writer, req.Body) if err != nil { // If something went wrong with copying data, abort the upload and return an error. writer.Abort() errs.HTTPError(w, err) return } if err := writer.Close(); err != nil { errs.HTTPError(w, err) return } // All good! Return a 200 OK. w.WriteHeader(http.StatusOK) } ``` ## Downloading files To download a file from a bucket, use the `Download` method on the bucket variable. It returns a reader that you can use to read the contents of the file. The `Download` method additionally takes a set of options to configure the download, like downloading a specific version if the bucket is versioned (`objects.WithVersion`). See the [package documentation](https://pkg.go.dev/encore.dev/storage/objects#Bucket.Download) for more details. For example, to download the user's profile picture and serve it: ```go package user import ( "context" "io" "net/http" "encore.dev" "encore.dev/beta/auth" "encore.dev/beta/errs" "encore.dev/storage/objects" ) var ProfilePictures = objects.NewBucket("profile-pictures", objects.BucketConfig{}) //encore:api public raw method=GET path=/profile-picture/:userID func ServeProfilePicture(w http.ResponseWriter, req *http.Request) { userID := encore.CurrentRequest().PathParams.Get("userID") reader := ProfilePictures.Download(req.Context(), userID) // Did we encounter an error? if err := reader.Err(); err != nil { errs.HTTPError(w, err) return } // Assuming all images are JPEGs. w.Header().Set("Content-Type", "image/jpeg") io.Copy(w, reader) } ``` ## Listing objects To list objects in a bucket, use the `List` method on the bucket variable. It returns an iterator of `(*objects.ListEntry, error)` pairs that you can use to easily iterate over the objects in the bucket using a `range` loop. For example, to list all profile pictures: ```go for entry, err := range ProfilePictures.List(ctx, &objects.Query{}) { if err != nil { // Handle error } // Do something with entry } ``` The `*objects.Query` type can be used to limit the number of objects returned, or to filter them to a specific key prefix. See the [package documentation](https://pkg.go.dev/encore.dev/storage/objects#Bucket.List) for more details. ## Deleting objects To delete an object from a bucket, use the `Remove` method on the bucket variable. For example, to delete a profile picture: ```go err := ProfilePictures.Remove(ctx, "my-user-id") if err != nil && !errors.Is(err, objects.ErrObjectNotFound) { // Handle error } ``` ## Retrieving object attributes You can retrieve information about an object using the `Attrs` method on the bucket variable. It returns the attributes of the object, like its size, content type, and ETag. For example, to get the attributes of a profile picture: ```go attrs, err := ProfilePictures.Attrs(ctx, "my-user-id") if errors.Is(err, objects.ErrObjectNotFound) { // Object not found } else if err != nil { // Some other error } // Do something with attrs ``` For convenience there is also `Exists` which returns a boolean indicating whether the object exists. ```go exists, err := ProfilePictures.Exists(ctx, "my-user-id") if err != nil { // Handle error } else if !exists { // Object does not exist } ``` ## Using Public Buckets Encore supports creating public buckets where objects can be accessed directly via HTTP/HTTPS without authentication. This is useful for serving static assets like images, videos, or other public files. To create a public bucket, set `Public: true` in the `BucketConfig`: ```go var PublicAssets = objects.NewBucket("public-assets", objects.BucketConfig{ Public: true, }) ``` Once configured as public, you can get the public URL for any object using the `PublicURL` method: ```go // Get the public URL for an object url := PublicAssets.PublicURL("path/to/image.jpg") // The URL can be used directly or shared publicly fmt.Println(url) // e.g. https://assets.example.com/path/to/image.jpg ``` When self-hosting, see how to configure public buckets in the [infrastructure configuration docs](https://encore.dev/docs/self-host/configure-infra). When deploying with the Encore platform it will automatically configure the bucket to be publicly accessible and [configure CDN](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) for optimal content delivery. ### Using bucket references Encore uses static analysis to determine which services are accessing each bucket, and what operations each service is performing. That information is used to provision infrastructure correctly, render architecture diagrams, and configure IAM permissions. This means that `*objects.Bucket` variables can't be passed around however you'd like, as it makes static analysis impossible in many cases. To work around these restrictions Encore allows you to get a "reference" to a bucket that can be passed around any way you want by calling `objects.BucketRef`. To ensure Encore still is aware of which permissions each service needs, the call to `objects.BucketRef` must be made from within a service. Additionally, it must pre-declare the permissions it needs; those permissions are then assumed to be used by the service. It looks like this (using the `ProfilePictures` topic above): ```go ref := objects.BucketRef[objects.Downloader](ProfilePictures) // ref is of type objects.Downloader, which allows downloading. ``` Encore provides permission interfaces for each operation that can be performed on a bucket: * `objects.Downloader` for downloading objects * `objects.Uploader` for uploading objects * `objects.Lister` for listing objects * `objects.Attrser` for getting object attributes * `objects.Remover` for removing objects * `objects.SignedDownloader` for generating signed download URLs for objects * `objects.SignedUploader` for generating signed upload URLs for objects If you need multiple permissions they can be combined by creating an interface that embeds the permissions you need. ```go type myPerms interface { objects.Downloader objects.Uploader } ref := objects.BucketRef[myPerms](ProfilePictures) ``` For convenience Encore provides an `objects.ReadWriter` interface that gives complete read-write access with all the permissions above. See the [package documentation](https://pkg.go.dev/encore.dev/storage/objects#BucketRef) for more details. ## Signed Upload URLs You can use `SignedUploadURL` to create signed URLs to allow clients to upload content directly into the bucket over the internet. The URL is always restricted to one filename, and has a set expiration date. Anyone in possession of the URL can upload data under this filename without any additional authentication. ```go url, err := ProfilePictures.SignedUploadURL(ctx, "my-user-id", objects.WithTTL(time.Duration(7200)*time.Second)) // Pass url to client ``` The client can now `PUT` to this URL with the content as a binary payload. ```bash curl -X PUT --data-binary @/home/me/dog-wizard.jpeg "https://storage.googleapis.com/profile-pictures/my-user-id/?x-goog-signature=b7a1<...>" ``` ### Why signed upload URLs? Signed URLs are an alternative to accepting the content payload directly in your API. Content upload requests are sometimes inconvenient to handle well: they can be long running and very large. With signed URLs, the content flows directly into the storage bucket, and only object IDs and metadata go through your API service. The trade-off is that the upload flow becomes more complex from a client point of view. ## Signed Download URLs You can use `SignedDownloadURL` to create signed URLs to allow clients to download content directly from the bucket, even if it's private. The URL is always restricted to one filename, and has a set expiration date. Anyone in possession of the URL can download the file without any additional authentication. ```go url, err := Documents.SignedDownloadURL(ctx, "letter-1234", objects.WithTTL(time.Duration(7200)*time.Second)) // Pass url to client ``` ### Why signed download URLs? Similar to the upload case, signed download URLs is a way to avoid handing large files or bulk traffic through your API. With signed URLs, the content flows directly from the storage bucket, and only object IDs and metadata go through your API service. Note: unless the content is private, prefer serving urls with `PublicURL()` over signed URLs. Public URLs go over CDN, which is typically significantly more performant and cost effective. --- Source: https://encore.dev/docs/go/how-to/pubsub-outbox Section: Building (Encore.go) # Transactional Pub/Sub outbox > Guarantee consistency between your database and Pub/Sub subscribers One of the hardest parts of building an event-driven application is ensuring consistency between services. A common pattern is for each service to have its own database and use Pub/Sub to notify other systems of business events. Inevitably this leads to inconsistencies since the Pub/Sub publishing is not transactional with the database writes. While there are several approaches to solving this, it's important the solution doesn't add too much complexity to what is often an already complex architecture. Perhaps the best solution in this regard is the [transactional outbox pattern](https://softwaremill.com/microservices-101/). Encore provides support for the transactional outbox pattern in the [x.encore.dev/infra/pubsub/outbox](https://pkg.go.dev/x.encore.dev/infra/pubsub/outbox) package. The transactional outbox works by binding a Pub/Sub topic to a database transaction, translating all calls to `topic.Publish` into inserting a database row in an `outbox` table. If/when the transaction later commits, the messages are picked up by a [Relay](https://pkg.go.dev/x.encore.dev/infra/pubsub/outbox#Relay) that polls the `outbox` table and publishes the messages to the actual Pub/Sub topic. ## Publishing messages to the outbox To publish messages to the outbox, a topic must first be bound to the outbox. This is done using [Pub/Sub topic references](https://encore.dev/docs/go/primitives/pubsub#using-topic-references) which allows you to retain complete type safety and the same interface as regular Pub/Sub topics, allowing existing code to continue to work without changes. > **Note** > > In regular (non-outbox) usage the message id returned by `topic.Publish` is the same as the message id the subscriber > receives when processing the message. With the outbox, this message id is not available until the transaction commits, > so `topic.Publish` returns an id referencing the outbox row instead. The topic binding supports pluggable storage backends, enabling use of the outbox pattern with any transactional storage backend. Implementation are provided out-of-the-box for use with Encore's `encore.dev/storage/sqldb` package, as well as the standard library `database/sql` and `github.com/jackc/pgx/v5` drivers, but it's easy to write your own for other use cases. See the [Go package reference](https://pkg.go.dev/x.encore.dev/infra/pubsub/outbox#PersistFunc) for more information. For example, to use a transactional outbox to notify subscribers when a user is created: ```go -- outbox.go -- // Create a SignupsTopic somehow. var SignupsTopic = pubsub.NewTopic[*SignupEvent](https://encore.dev/* ... */) // Create a topic ref with publisher permissions. ref := pubsub.TopicRef[pubsub.Publisher[*SignupEvent]](SignupsTopic) // Bind it to the transactional outbox import "x.encore.dev/infra/pubsub/outbox" var tx *sqldb.Tx // somehow get a transaction ref = outbox.Bind(ref, outbox.TxPersister(tx)) // Calls to ref.Publish() will now insert a row in the outbox table. -- db_migration.sql -- -- The database used must contain the below database table: -- See https://pkg.go.dev/x.encore.dev/infra/pubsub/outbox#SQLDBStore CREATE TABLE outbox ( id BIGSERIAL PRIMARY KEY, topic TEXT NOT NULL, data JSONB NOT NULL, inserted_at TIMESTAMPTZ NOT NULL ); CREATE INDEX outbox_topic_idx ON outbox (topic, id); ``` Once the transaction commits any published messages via `ref` above will be stored in the `outbox` table. ## Consuming messages from the outbox Once committed, the messages are ready to be picked up and published to the actual Pub/Sub topic. That is done via the [Relay](https://pkg.go.dev/x.encore.dev/infra/pubsub/outbox#Relay). The relay continuously polls the `outbox` table and publishes any new messages to the actual Pub/Sub topic. The relay supports pluggable storage backends, enabling use of the outbox pattern with any transactional storage backend. An implementation is provided out-of-the-box that uses Encore's built-in [SQL database support](https://pkg.go.dev/x.encore.dev/infra/pubsub/outbox#SQLDBStore), but it's easy to write your own for other databases. The topics to poll must be registered with the relay, typically during service initialization. For example: ```go -- user/service.go -- package user import ( "context" "encore.dev/pubsub" "encore.dev/storage/sqldb" "x.encore.dev/infra/pubsub/outbox" ) //encore:service type Service struct { signupsRef pubsub.Publisher[*SignupEvent] } // db is the database the outbox table is stored in var db = sqldb.NewDatabase(...) // Create the SignupsTopic somehow. var SignupsTopic = pubsub.NewTopic[*SignupEvent](https://encore.dev/* ... */) func initService() (*Service, error) { // Initialize the relay to poll from our database. relay := outbox.NewRelay(outbox.SQLDBStore(db)) // Register the SignupsTopic to be polled. signupsRef := pubsub.TopicRef[pubsub.Publisher[*SignupEvent]](SignupsTopic) outbox.RegisterTopic(relay, signupsRef) // Start polling. go relay.PollForMessage(context.Background(), -1) return &Service{signupsRef: signupsRef}, nil } ``` --- Source: https://encore.dev/docs/go/primitives/pubsub Section: Building (Encore.go) # Pub/Sub > Decoupling services and building asynchronous systems Publishers & Subscribers (Pub/Sub) let you build systems that communicate by broadcasting events asynchronously. This is a great way to decouple services for better reliability and responsiveness. Encore's Backend Framework lets you use Pub/Sub in a cloud-agnostic declarative fashion. At deployment, Encore automatically [provisions the required infrastructure](https://encore.dev/docs/platform/infrastructure/infra). ## Creating a Topic The core of Pub/Sub is the **Topic**, a named channel on which you publish events. Topics must be declared as package level variables, and cannot be created inside functions. Regardless of where you create a topic, it can be published to from any service, and subscribed to from any service. When creating a topic, it must be given an event type, a unique name, and a configuration to define its behaviour. See the complete specification in the [package documentation](https://pkg.go.dev/encore.dev/pubsub#NewTopic). For example, to create a topic with events about user signups: ```go package user import "encore.dev/pubsub" type SignupEvent struct{ UserID int } var Signups = pubsub.NewTopic[*SignupEvent]("signups", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) ``` **Related example:** [encoredev/examples/uptime](https://github.com/encoredev/examples/tree/main/uptime) Event-driven example application using Pub/Sub. Create it locally with `encore app create --example=uptime`. ### At-least-once delivery The above example configures the topic to ensure that, for each subscription, events will be delivered _at least once_. This means that if the topic believes the event was not processed, it will attempt to deliver the message again. **Therefore, all subscription handlers should be [idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning).** This helps ensure that if the handler is called two or more times, from the outside there's no difference compared to calling it once. This can be achieved using a database to track if you have already performed the action that the event is meant to trigger, or ensuring that the action being performed is also idempotent in nature. ### Exactly-once delivery Topics can also be configured to deliver events _exactly once_ by setting the `DeliveryGuarantee` field to `pubsub.ExactlyOnce`. This enables stronger guarantees on the infrastructure level to minimize the likelihood of message re-delivery. However, there are still some rare circumstances when a message might be redelivered. For example, if a networking issue causes the acknowledgement of successful processing the message to be lost before the cloud provider receives it (the [Two Generals' Problem](https://en.wikipedia.org/wiki/Two_Generals%27_Problem)). As such, if correctness is critical under all circumstances, it's still advisable to design your subscription handlers to be idempotent. By enabling exactly-once delivery on a topic the cloud provider enforces certain throughput limitations: - AWS: 300 messages per second for the topic (see [AWS SQS Quotas](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html)). - GCP: At least 3,000 messages per second across all topics in the region (can be higher on the region see [GCP PubSub Quotas](https://cloud.google.com/pubsub/quotas#quotas)). > **Important** > > Exactly-once delivery does not perform message deduplication on the publishing side. If `Publish` is called twice with > the same message, the message will be delivered twice. ### Ordered Topics Topics are unordered by default, meaning that messages can be delivered in any order. This allows for better throughput on the topic as messages can be processed in parallel. However, in some cases, messages must be delivered in the order they were published for a given entity. To create an ordered topic, configure the topic's `OrderingAttribute` to match the `pubsub-attr` tag on one of the top-level fields of the event type. This field ensures that messages delivered to the same subscriber are delivered in the order of publishing for that specific field value. Messages with a different value on the ordering attribute are delivered in an unspecified order. To maintain topic order, messages with the same ordering key aren't delivered until the earliest message is processed or dead-lettered, potentially causing delays due to [head-of-line blocking](https://en.wikipedia.org/wiki/Head-of-line_blocking). Mitigate processing issues by ensuring robust logging and alerts, and appropriate subscription retry policies. > **Note** > > The `OrderingAttribute` currently has no effect in local environments. #### Throughput limitations Each cloud provider enforces certain throughput limitations for ordered topics: - **AWS:** 300 messages per second for the topic (see [AWS SQS Quotas](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html)) - **GCP:** 1 MBps for each ordering key (See [GCP Pub/Sub Resource Limits](https://cloud.google.com/pubsub/quotas#resource_limits)) #### Ordered topic example ```go package example import ( "context" "encore.dev/pubsub" ) type CartEvent struct { ShoppingCartID int `pubsub-attr:"cart_id"` Event string } var CartEvents = pubsub.NewTopic[*CartEvent]("cart-events", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, OrderingAttribute: "cart_id", }) func Example(ctx context.Context) error { // These are delivered in order as they all have the same shopping cart ID CartEvents.Publish(ctx, &CartEvent{ShoppingCartID: 1, Event: "item_added"}) CartEvents.Publish(ctx, &CartEvent{ShoppingCartID: 1, Event: "checkout_started"}) CartEvents.Publish(ctx, &CartEvent{ShoppingCartID: 1, Event: "checkout_completed"}) // This event may be delivered at any point as it has a different shopping cart ID CartEvents.Publish(ctx, &CartEvent{ShoppingCartID: 2, Event: "item_added"}) } ``` ## Publishing events To publish an **Event**, call `Publish` on the topic passing in the event object (which is the type specified in the `pubsub.NewTopic[Type]` constructor). For example: ```go messageID, err := Signups.Publish(ctx, &SignupEvent{UserID: id}) if err != nil { return err } // If we get here the event has been successfully published, // and all registered subscribers will receive the event. // The messageID variable contains the unique id of the message, // which is also provided to the subscribers when processing the event. ``` By defining the `Signups` topic variable as an exported variable you can also publish to the topic from other services in the same way. ### Using topic references Encore uses static analysis to determine which services are publishing messages to what topics. That information is used to provision infrastructure correctly, render architecture diagrams, and configure IAM permissions. This means that `*pubsub.Topic` variables can't be passed around however you'd like, as it makes static analysis impossible in many cases. To work around these restrictions Encore allows you to get a reference to a topic that can be passed around any way you want. It looks like this (using the `Signups` topic above): ```go signupRef := pubsub.TopicRef[pubsub.Publisher[*SignupEvent]](Signups) // signupRef is of type pubsub.Publisher[*SignupEvent], which allows publishing. ``` The difference between a **TopicRef** and a **Topic** is that topic references need to pre-declare what permissions are needed. Encore then assumes that all the permissions you declare are used. For example, if you declare a **TopicRef** with the `pubsub.Publisher` permission (as seen above) Encore assumes that the service will publish messages to the topic and provisions the infrastructure to support that. Note that a **TopicRef** must be declared _within a service_, but the reference itself can be freely passed around to library code, be dependency injected into [service structs](https://encore.dev/docs/go/how-to/dependency-injection), and so on. ## Subscribing to Events To **Subscribe** to events, you create a Subscription as a package level variable by calling the [`pubsub.NewSubscription`](https://pkg.go.dev/encore.dev/pubsub#NewSubscription) function. Each subscription needs: - the topic to subscribe to - a name which is unique for the topic - a configuration object with at least a `Handler` function to process the events - a configuration object Here's an example of how you create a subscription to a topic: ```go package email import ( "encore.dev/pubsub" "user" ) var _ = pubsub.NewSubscription( user.Signups, "send-welcome-email", pubsub.SubscriptionConfig[*SignupEvent]{ Handler: SendWelcomeEmail, }, ) func SendWelcomeEmail(ctx context.Context, event *SignupEvent) error { // send email... return nil } ``` Subscriptions can be in the same service as the topic is declared, or in any other service of your application. Each subscription to a single topic receives the events independently of any other subscriptions to the same topic. This means that if one subscription is running very slowly, it will grow a backlog of unprocessed events. However, any other subscriptions will still be processing events in real-time as they are published. The `ctx` passed to the handler function is cancelled when the `AckDeadline` for the subscription is reached. This is the time when the message is considered to have timed out and can be redelivered to another subscriber. The timeout defaults to 30 seconds if you don't explicitly configure `AckDeadline`. ### Method-based handlers When using [service structs](https://encore.dev/docs/go/primitives/service-structs) for dependency injection it's common to want to define the subscription handler as a method on the service struct, to be able to access the injected dependencies. The pubsub package provides the `pubsub.MethodHandler` function for this purpose: ```go //encore:service type Service struct { /* ... */ } func (s *Service) SendWelcomeEmail(ctx context.Context, event *SignupEvent) error { // ... } var _ = pubsub.NewSubscription( user.Signups, "send-welcome-email", pubsub.SubscriptionConfig[*SignupEvent]{ Handler: pubsub.MethodHandler((*Service).SendWelcomeEmail), }, ) ``` Note that `pubsub.MethodHandler` only allows referencing methods on the service struct type, not any other type. ### Subscription configuration When creating a subscription you can configure behavior such as message retention and retry policy, using the `SubscriptionConfig` type. See the [package documentation](https://pkg.go.dev/encore.dev/pubsub#SubscriptionConfig) for the complete configuration options. > **Note** > > The `SubscriptionConfig` struct fields must be defined as compile-time constants, and cannot be defined in > terms of function calls. This is necessary for Encore to understand the exact requirements of the subscription, in order to provision the correct infrastructure upon deployment. ### Error Handling If a subscription function returns an error, the event being processed will be retried, based on the retry policy [configured on that subscription](https://pkg.go.dev/encore.dev/pubsub#SubscriptionConfig). After the `MaxRetries` is hit, the event will be placed into a dead-letter queue (DLQ) for that subscriber. This allows the subscription to continue processing events until the bug which caused the event to fail can be fixed. Once fixed, the messages on the dead-letter queue can be manually released to be processed again by the subscriber. ## Testing Pub/Sub Encore uses a special testing implementation of Pub/Sub topics. When running tests, topics are aware of which test is running. This gives you the following guarantees: - Your subscriptions will not be triggered by events published. This allows you to test the behaviour of publishers independently of side effects caused by subscribers. - Message ID's generated on publish are deterministic (based on the order of publishing), thus your assertions can make use of that fact. - Each test is isolated from other tests, meaning that events published in one test will not impact other tests (even if you use parallel testing). Encore provides a helper function, [`et.Topic`](https://pkg.go.dev/encore.dev/et#Topic), to access the testing topic. You can use this object to extract the events that have been published to it during a test. Here's an example implementation: ```go package user import ( "testing" "encore.dev/et" "github.com/stretchr/testify/assert" ) func Test_Register(t *testing.T) { t.Parallel() ... Call Register() and assert changes to the database ... // Get all published messages on the Signups topic from this test. msgs := et.Topic(Signups).PublishedMessages() assert.Len(t, msgs, 1) } ``` ## Ensuring consistency between services Ensuring consistency between services in event-driven applications can be challenging, especially when database writes and Pub/Sub publishing are not transactional. This can lead to inconsistencies between services. To address this issue without adding excessive complexity, consider using a transactional outbox pattern. For more information on implementing this pattern with Encore, see the [Pub/Sub Outbox guide](https://encore.dev/docs/go/how-to/pubsub-outbox). ## The benefits of Pub/Sub Pub/Sub is a powerful building block in a backend application. It can be used to improve app reliability by reducing the blast radius of faulty components and bottlenecks. It can also be used to increase the speed of response to the user, and even helps reduce cognitive overhead for developers by inverting the dependencies between services. For those not familiar with Pub/Sub, lets take a look at an example API in a user registration service. The behavior we want to implement is that upon registration, we send a welcome email to the user and create a record of the signup in our analytics system. Now let's see how we could implement this only using APIs, compared to how a Pub/Sub implementation might look. ### An API only approach Using API calls between services, we might design a system which looks like this when the user registers: ![](https://encore.dev/assets/docs/pubsub-rpc-example.png) 1. The `user` service starts a database transaction and records the user in its database. 2. The `user` service makes a call to the `email` service to send a welcome email. 3. The `email` service then calls an email provider to actually send the email. 4. Upon success, the `email` service replies to the `user` service that the request was processed. 5. The `user` service then calls the `analytics` service to record the signup. 6. The `analytics` service the writes to the data warehouse to record the information. 7. The `analytics` service then replies to the `user` service that the request was processed. 8. The `user` service commits the database transaction. 9. The `user` service then can reply to the user to say the registration was successful. Notice how we have to wait for everything to complete before we can reply to the user to tell then we've registered them. This means that if our email provider takes 3 seconds to send the email, we've now taken 3 seconds to respond to the user, when in reality once the user was written to the database, we could have responded to the user instantly at that point to confirm the registration. Another downside to this approach is if our data warehouse is currently broken and reporting errors, our system will also report errors whenever anybody tries to signup! Given analytics is purely internal and doesn't impact users, why should the analytics system being down impact user signup? ### A Pub/Sub approach A more ideal solution would be if we could decouple the behaviour of emailing the user and recording our analytics, such that the user service only has to record the user in its own database and let the user know they are registered - without worrying about the downstream impacts. Thankfully, this is exactly what [Pub/Sub topics](https://pkg.go.dev/encore.dev/pubsub#Topic) allow us to do. In this example, when a user registers we: 1. The `user` service starts a database transaction and records the user in its database. 2. Publish a signup event to the `signups` topic. 3. Commit the transaction and reply to the user to say the registration was successful. At this point the user is free to continue interacting with the application and we've isolated the registration behaviour from the rest of the application. In parallel, the `email` and `analytics` services will receive the signup event from the `signups` topic and will then perform their respective tasks. If either service returns an error, the event will automatically be backed off and retried until the service is able to process the event successfully, or reaches the maximum number of attempts and is placed into the deadletter queue (DLQ). ![](https://encore.dev/assets/docs/pubsub-topic-example.png) Notice how in this version, the processing time of the two other services did not impact the end user and in fact the `user` service is not even aware of the `email` and `analytics` services. This means that new systems which need to know about new users signing up can be added to the application, without the need to change the `user` service or impacting its performance. --- Source: https://encore.dev/docs/go/primitives/cron-jobs Section: Building (Encore.go) # Cron Jobs > Run recurring and scheduled tasks When you need to run periodic and recurring tasks, Encore.go provides a declarative way of using Cron Jobs. When a Cron Job is defined in your application, Encore automatically calls your specified API according to the defined schedule. This eliminates the need for infrastructure maintenance, as Encore manages scheduling, monitoring, and execution of Cron Jobs. > **Note** > > Cron Jobs do not run when developing locally or in [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments), but you can always call the API manually to test the behavior. **Related example:** [encoredev/examples/uptime](https://github.com/encoredev/examples/tree/main/uptime) Uptime Monitoring app that uses a Cron Job to periodically check the uptime of a website. Create it locally with `encore app create --example=uptime`. ## Defining a Cron Job To define a Cron Job, import the `encore.dev/cron` [package](https://pkg.go.dev/encore.dev/cron), and call the `cron.NewJob()` function and store it as a package-level variable. ### Example ```go import "encore.dev/cron" // Send a welcome email to everyone who signed up in the last two hours. var _ = cron.NewJob("welcome-email", cron.JobConfig{ Title: "Send welcome emails", Every: 2 * cron.Hour, Endpoint: SendWelcomeEmail, }) // SendWelcomeEmail emails everyone who signed up recently. // It's idempotent: it only sends a welcome email to each person once. //encore:api private func SendWelcomeEmail(ctx context.Context) error { // ... return nil } ``` The `"welcome-email"` argument to `cron.NewJob` is a unique ID you give to each Cron Job. If you later refactor the code and move the Cron Job definition to another package, we use this ID to keep track that it's the same Cron Job and not a different one. When this code gets deployed Encore will automatically register the Cron Job in the Encore platform and begin calling the `SendWelcomeEmail` API every hour. The Encore platform dashboard provides a convenient user interface for monitoring and debugging Cron Job executions across all your environments via the `Cron Jobs` menu item: ![Cron Jobs UI](https://encore.dev/assets/docs/cron.png) ## Keep in mind when using Cron Jobs - Cron Jobs do not execute during local development or in [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments). However, you can manually invoke the API to test its behavior. - In the Encore platform, Cron Job executions are limited to **once every hour**, with the exact minute randomized within that hour for users on the Free Tier. To enable more frequent executions or to specify the exact minute within the hour, consider [deploying to your own cloud](https://encore.dev/docs/platform/deploy/own-cloud) or upgrading to the [Pro plan](https://encore.dev/pricing). - Both public and private APIs are supported for Cron Jobs. - Ensure that the API endpoints used in Cron Jobs are idempotent, as they may be called multiple times under certain network conditions. - The API endpoints used in Cron Jobs must not take any request parameters. That is, their signatures must be `func(context.Context) error` or `func(context.Context) (*T, error)`. ## Cron schedules Above we used the `Every` field, which executes the Cron Job on a periodic basis. It runs around the clock each day, starting at midnight (UTC). In order to ensure a consistent delay between each run, the interval used **must divide 24 hours evenly**. For example, `10 * cron.Minute` and `6 * cron.Hour` are both allowed (since 24 hours is evenly divisible by both), whereas `7 * cron.Hour` is not (since 24 is not evenly divisible by 7). The Encore compiler will catch this and give you a helpful error at compile-time if you try to use an invalid interval. ### Cron expressions For more advanced use cases, such as running a Cron Job on a specific day of the month, or a specific week day, or similar, the `Every` field is not expressive enough. For these use cases, Encore provides full support for [Cron expressions](https://en.wikipedia.org/wiki/Cron) by using the `Schedule` field instead of the `Every` field. Cron expressions allow you to define precise schedules for your tasks, including specific days of the week, specific hours of the day, and more. Note that all times are expressed in UTC. For example: ```go // Run the monthly accounting sync job at 4am (UTC) on the 15th day of each month. var _ = cron.NewJob("accounting-sync", cron.JobConfig{ Title: "Cron Job Example", Schedule: "0 4 15 * *", Endpoint: AccountingSync, }) ``` --- Source: https://encore.dev/docs/go/primitives/caching Section: Building (Encore.go) # Caching > Optimize response times and reduce costs by avoiding re-work A cache is a high-speed storage layer, commonly used in distributed systems to improve user experiences by reducing latency, improving system performance, and avoiding expensive computation. For scalable systems you typically want to deploy the cache as a separate infrastructure resource, allowing you to run multiple instances of your application concurrently. Encore's built-in Caching API lets you use high-performance caches (using [Redis](https://redis.io/)) in a cloud-agnostic declarative fashion. At deployment, Encore will automatically [provision the required infrastructure](https://encore.dev/docs/platform/infrastructure/infra). ## Cache clusters To use caching in Encore, you must first define a *cache cluster*. Each cache cluster defined in your application will be provisioned as a separate Redis instance by Encore. This gives you fine-grained control over which service(s) should use the same cache cluster and which should have a separate one. It looks like this: ```go import "encore.dev/storage/cache" var MyCacheCluster = cache.NewCluster("my-cache-cluster", cache.ClusterConfig{ // EvictionPolicy tells Redis how to evict keys when the cache reaches // its memory limit. For typical cache use cases, cache.AllKeysLRU is a good default. EvictionPolicy: cache.AllKeysLRU, }) ``` > **Note** > > When starting out it's recommended to use a single cache cluster > that's shared between your different services. ## Keyspaces When using a cache, each cached item is stored at a particular key, which is typically an arbitrary string. If you use a cache cluster to cache different sets of data, it's important that distinct data set have non-overlapping keys. Each value stored in the cache also has a specific type, and certain cache operations can only be performed on certain types. For example, a common cache operation is to increment an integer value that is stored in the cache. If you try to apply this operation on a value that is not an integer, an error is returned. Encore provides a simple, type-safe solution to these problems through Keyspaces. In order to begin storing data in your cache, you must first define a Keyspace. Each keyspace has a Key type and a Value type. The Key type is much like a map key, in that it tells Encore where in the cache the item is stored. The Key type is combined with the Key Pattern to produce a string that is the Redis cache key. The Value type is the type of the values stored in that keyspace. For many keyspaces this is specified in the name of the constructor. For example, `NewIntKeyspace` stores `int64` values. For example, if you want to rate limit the number of requests per user ID it looks like this: ```go import ( "encore.dev/beta/auth" "encore.dev/beta/errs" "encore.dev/middleware" ) // RequestsPerUser tracks the number of requests per user. // The cache items expire after 10 seconds without activity. var RequestsPerUser = cache.NewIntKeyspace[auth.UID](cluster, cache.KeyspaceConfig{ KeyPattern: "requests/:key", DefaultExpiry: cache.ExpireIn(10 * time.Second), }) // RateLimitMiddleware is a global middleware that limits the number of authenticated requests // to 10 requests per 10 seconds. //encore:middleware target=all func RateLimitMiddleware(req middleware.Request, next middleware.Next) middleware.Response { if userID, ok := auth.UserID(); ok { val, err := RequestsPerUser.Increment(req.Context(), userID, 1) // NOTE: this "fails open", meaning if we can't communicate with the cache // we default to allowing the requests. // // Consider whether that's the correct behavior for your application, // or if you want to return an error to the user in that case. if err == nil && val > 10 { return middleware.Response{ Err: &errs.Error{Code: errs.ResourceExhausted, Message: "rate limit exceeded"}, } } } return next(req) } ``` As you can see, the `RequestsPerUser` defines a `KeyPattern` which is set to `"requests/:key"`. Here `:key` refers to the value of the Key type, which is the `auth.UID` value passed in. If you want the cache key to contain multiple values, you can define a struct type and pass that as the key. Then change the `KeyPattern` to specify the struct fields. For example: ```go type MyKey struct { UserID auth.UID ResourcePath string // the resource being accessed } // ResourceRequestsPerUser tracks the number of requests per user and resource. // The cache items expire after 10 seconds without activity. var ResourceRequestsPerUser = cache.NewIntKeyspace[MyKey](cluster, cache.KeyspaceConfig{ KeyPattern: "requests/:UserID/:ResourcePath", DefaultExpiry: cache.ExpireIn(10 * time.Second), }) // ... then: key := MyKey{UserID: "some-user-id", ResourcePath: "/foo"} ResourceRequestsPerUser.Increment(ctx, key, 1) ``` > **Note** > > Encore ensures that all the struct fields are present in the `KeyPattern`, > and that the placeholder values are all valid field names. > > That way the connection between the struct fields and the `KeyPattern` > become compile-time type-safe as well. Also note that Encore ensures there are no conflicting `KeyPattern` definitions across each cache cluster. Each keyspace must define its own, non-conflicting `KeyPattern`. This way, you can feel safe that there won't be any accidental overwrites of cache values, even with multiple services sharing the same cache cluster. ## Keyspace operations Encore comes with a full suite of keyspace types, each with a wide variety of cache operations. Basic keyspace types include [strings](https://pkg.go.dev/encore.dev/storage/cache#NewStringKeyspace), [integers](https://pkg.go.dev/encore.dev/storage/cache#NewIntKeyspace), [floats](https://pkg.go.dev/encore.dev/storage/cache#NewFloatKeyspace), and [struct types](https://pkg.go.dev/encore.dev/storage/cache#NewStructKeyspace). These keyspaces all share the same set of methods (along with a few keyspace-specific ones). There are also more advanced keyspaces for storing [sets of basic types](https://pkg.go.dev/encore.dev/storage/cache#NewSetKeyspace) and [ordered lists of basic types](https://pkg.go.dev/encore.dev/storage/cache#NewListKeyspace). These keyspaces offer a different, specialized set of methods specific to set and list operations. For a list of the supported operations, see the [package documentation](https://pkg.go.dev/encore.dev/storage/cache). ## Testing When running tests, Encore spins up an in-memory cache separately for each test. This way you don't have to think about clearing the cache between tests, or worrying about whether one test affects another. Each test is automatically fully isolated. ## Local development For local development, Encore maintains a local, in-memory implementation of Redis. This implementation is designed to store a small amount of keys (currently 100). When the number of keys exceeds this value, keys are randomly purged to get below the limit. This is designed in order to simulate the ephemeral, transient nature of caches while also limiting memory use. The precise behavior for local development may change over time and should not be relied on. --- Source: https://encore.dev/docs/go/primitives/secrets Section: Building (Encore.go) # Storing Secrets and API keys > Simply storing secrets securely Wouldn't it be nice to store secret values like API keys, database passwords, and private keys directly in the source code? Of course, we can’t do that – it's horrifyingly insecure! (Unfortunately, it's also [very common](https://www.ndss-symposium.org/ndss-paper/how-bad-can-it-git-characterizing-secret-leakage-in-public-github-repositories/).) Encore's built-in secrets manager makes it simple to store secrets in a secure way and lets you use them in your program like regular variables. **Related example:** [encoredev/examples/slack-bot](https://github.com/encoredev/examples/tree/main/slack-bot) Slack Bot example application using secrets to store a Slack key. Create it locally with `encore app create --example=slack-bot`. ## Using secrets in your application To use a secret in your application, first define it directly in your code by creating an unexported struct named `secrets`, where all fields are of type `string`. For example: ```go var secrets struct { SSHPrivateKey string // ed25519 private key for SSH server GitHubAPIToken string // personal access token for deployments // ... } ``` When running your application locally, if a secret is not set, it is left empty and you will get a warning notifying you that a secret value is missing. When deploying to a cloud environment, all secrets must be defined, otherwise the deploy will fail. Your app's default `staging` environment is a `development` environment, so set a value for it before you push, for example with `encore secret set --type dev,local GitHubAPIToken`. If a deploy fails with `secret key(s) not defined`, set the secret, then redeploy without a new commit: in the Cloud Dashboard, open **Deployments** > **New Deploy**, choose **Branch**, and select the branch the environment deploys from (usually `main`). Once you've provided values for all secrets, you can just use them in your application like a regular variable. For example: ```go func callGitHub(ctx context.Context) { req, _ := http.NewRequestWithContext(ctx, "GET", "https:///api.github.com/user", nil) req.Header.Add("Authorization", "token " + secrets.GitHubAPIToken) resp, err := http.DefaultClient.Do(req) // ... handle err and resp } ``` > **Note** > > Secret keys are globally unique for your whole application. If multiple services use the same secret name they both receive the same secret value at runtime. ## Storing secret values ### Using the Encore platform dashboard The simplest way to set up secrets is with the Secrets Manager in the Encore platform dashboard. Open your app in [app.encore.dev](https://app.encore.dev), go to **Settings** in the main navigation, and then click on **Secrets** in the settings menu. From here you can create secrets, save secret values, and configure different values for different environments. ![Encore's Secrets Manager](https://encore.dev/assets/docs/secrets.png) ### Using the CLI If you prefer, you can also set up secrets from the CLI using:
`encore secret set --type ` `` defines which environment types the secret value applies to. Use a comma-separated list of `production`, `development`, `preview`, and `local`. Shorthands: `prod`, `dev`, `pr`. For example `encore secret set --type prod SSHPrivateKey` sets the secret value for production environments,
and `encore secret set --type dev,preview,local GitHubAPIToken` sets the secret value for development, preview, and local environments. In some cases, it can be useful to define a secret for a specific environment instead of an environment type. You can do so with `encore secret set --env `. Secret values for specific environments take precedence over values for environment types. ### Environment settings Each secret can only have one secret value for each environment type. For example: If you have a secret value that's shared between `development`, `preview` and `local`, and you want to override the value for `local`, you must first edit the existing secret and remove `local` using the Secrets Manager in the [Encore platform dashboard](https://app.encore.dev). You can then add a new secret value for `local`. The end result should look something like the picture below. ![Overriding a secret in Encore's Secrets Manager](https://encore.dev/assets/docs/secretoverride.png) ## How it works: Where secrets are stored When you store a secret Encore stores it encrypted using Google Cloud Platform's [Key Management Service](https://cloud.google.com/security-key-management) (KMS). - **Production / Your own cloud:** When you deploy to production using your own cloud account on GCP or AWS, Encore provisions a secrets manager in your account (using either KMS or AWS Secrets Manager) and replicates your secrets to it. The secrets are then injected into the container using secret environment variables. - **Local:** For local secrets Encore automatically replicates them to developers' machines when running `encore run`. - **Development / the Encore platform:** Environments on Encore's development cloud (running on GCP under the hood) work the same as self-hosted GCP environments, using GCP Secrets Manager. ### Overriding local secrets When setting secrets via the `encore secret set` command, they are automatically synced to all developers working on the same application, courtesy of the Encore platform. In some cases, however, you want to override a secret only for your local machine. This can be done by creating a file named `.secrets.local.cue` in the root of your Encore application, next to the `encore.app` file. The file contains key-value pairs of secret names to secret values. For example: ```cue -- .secrets.local.cue -- GitHubAPIToken: "my-local-override-token" SSHPrivateKey: "custom-ssh-private-key" ``` --- Source: https://encore.dev/docs/go/develop/metadata Section: Building (Encore.go) # Metadata > Use the metadata API to get specifics about apps, environments, and requests While Encore tries to provide a cloud-agnostic environment, sometimes it's helpful to know more about the environment your application is running in. For this reason Encore provides an API for accessing metadata about the [application](#application-metadata) and the environment it's running in, as well as information about the [current request](#current-request) as part of the `encore.dev` package. ## Application Metadata Calling `encore.Meta()` will return an [encore.AppMetadata](https://pkg.go.dev/encore.dev/#AppMetadata) instance which contains information about the application, including: - `AppID` - the application name. - `APIBaseURL` - the URL the application API can be publicly accessed on. - `Environment` - the [environment](https://encore.dev/docs/platform/deploy/environments) the application is currently running in. - `Build` - the revision information of the build from the version control system. - `Deploy` - the deployment ID and when this version of the app was deployed. ## Current Request `encore.CurrentRequest()` can be called from anywhere within your application and will return an [encore.Request](https://pkg.go.dev/encore.dev/#Request) instance which will provides information about why the current code is running. The [encore.Request](https://pkg.go.dev/encore.dev/#Request) type contains information about the running request, such as: - The service and endpoint being called - Path and path parameter information - When the request started This works automatically as a result of Encore's request tracking, and works even in other goroutines that were spawned during request handling. If no request is processed by the caller, which can happen if you call it during service initialization, the Type field returns None. If `CurrentRequest()` is called from a goroutine spawned during request processing it will continue to report the same request even if the request handler has already returned. This can be useful on [raw endpoints](https://encore.dev/docs/go/primitives/raw-endpoints) with [path parameters](https://encore.dev/docs/go/primitives/defining-apis#rest-apis) as the standard `http.Request` object passed into the raw endpoint does not provide access to the parsed path parameters, however by calling `encore.CurrentRequest().PathParams()` you can get access to the parsed path parameters. ## Example Use Cases ### Using Cloud Specific Services All the [clouds](https://encore.dev/docs/platform/deploy/own-cloud) contain a large number of services, not all of which Encore natively supports. By using information about the [environment](https://encore.dev/docs/platform/deploy/environments), you can define the implementation of these and use different services for each environment's provider. For instance if you are pushing audit logs into a data warehouse, when running on GCP you could use BigQuery, but when running on AWS you could use Redshift, when running locally you could simply write them to a file. ```go package audit import ( "encore.dev" "encore.dev/beta/auth" ) func Audit(ctx context.Context, action message, user auth.UID) error { switch encore.Meta().Environment.Cloud { case encore.CloudAWS: return writeIntoRedshift(ctx, action, user) case encore.CloudGCP: return writeIntoBigQuery(ctx, action, user) case encore.CloudLocal: return writeIntoFile(ctx, action, user) default: return fmt.Errorf("unknown cloud: %s", encore.Meta().Environment.Cloud) } } ``` ### Checking Environment type When implementing a signup system, you may want to skip email verification on user signups when developing the application. Using the `encore.Meta()` API, we can check the environment and decide whether to send an email or simply mark the user as verified upon signup. ```go package user import "encore.dev" //encore:api public func Signup(ctx context.Context, params *SignupParams) (*SignupResponse, error) { // ... // If this is a testing environment, skip sending the verification email switch encore.Meta().Environment.Type { case encore.EnvTest, encore.EnvDevelopment: if err := MarkEmailVerified(ctx, userID); err != nil { return nil, err } default: if err := SendVerificationEmail(ctx, userID); err != nil { return nil, err } } // ... } ``` --- Source: https://encore.dev/docs/go/develop/testing/mocking Section: Building (Encore.go) # Mocking > Testing your application in isolation Encore comes with built-in support for mocking out APIs and services, which makes it easier to test your application in isolation. ## Mocking Endpoints Let's say you have an endpoint that calls an external API in our `products` service: ```go //encore:api private func GetPrice(ctx context.Context, p *PriceParams) (*PriceResponse, error) { // Call external API to get the price } ``` When testing this function, you don't want to call the real external API since that would be slow and cause your tests to fail if the API is down. Instead, you want to mock out the API call and return a fake response. In Encore, you can do this by adding a mock implementation of the endpoint using the `et.MockEndpoint` function inside your test: ```go package shoppingcart import ( "context" "testing" "encore.dev/et" // Encore's test support package "your_app/products" ) func Test_Something(t *testing.T) { t.Parallel() // Run this test in parallel with other tests without the mock implementation interfering // Create a mock implementation of pricing API which will only impact this test and any sub-tests et.MockEndpoint(products.GetPrice, func(ctx context.Context, p *products.PriceParams) (*products.PriceResponse, error) { return &products.PriceResponse{Price: 100}, nil }) // ... the rest of your test code here ... } ``` When any code within the test, or any sub-test calls the `GetPrice` API, the mock implementation will be called instead. The mock will not impact any other tests running in parallel. The function you pass to `et.MockEndpoint` must have the same signature as the real endpoint. If you want to mock out the API for all tests in the package, you can add the mock implementation to the `TestMain` function: ```go package shoppingcart import ( "context" "os" "testing" "encore.dev/et" "your_app/products" ) func TestMain(m *testing.M) { // Create a mock implementation of pricing API which will impact all tests within this package et.MockEndpoint(products.GetPrice, func(ctx context.Context, p *products.PriceParams) (*products.PriceResponse, error) { return &products.PriceResponse{Price: 100}, nil }) // Now run the tests os.Exit(m.Run()) } ``` Mocks can be changed at any time, including removing them by setting the mock implementation to `nil`. ## Mocking services As well as mocking individual APIs, you can also mock entire services. This can be useful if you want to inject a different set of dependencies into your service for testing, or a service that your code depends on. This can be done using the `et.MockService` function: ```go package shoppingcart import ( "context" "testing" "encore.dev/et" // Encore's test support package "your_app/products" ) func Test_Something(t *testing.T) { t.Parallel() // Run this test in parallel with other tests without the mock implementation interfering // Create a instance of the products service which will only impact this test and any sub-tests et.MockService("products", &products.Service{ SomeField: "a testing value", }) // ... the rest of your test code here ... } ``` When any code within the test, or any sub-test calls the `products` service, the mock implementation will be called instead. Unlike `et.MockEndpoint`, the mock implementation does not need to have the same signature, and can be any object. The only requirement is that any of the services APIs that are called during the test must be implemented by as a receiver method on the mock object. (This also includes APIs that are defined as package level functions in the service, and are not necessarily defined as receiver methods on that services struct). To help with compile time safety on service mocking, for every service Encore will automatically generate an `Interface` interface which contains all the APIs defined in the service. This interface can be passed as a generic argument to `et.MockService` to ensure that the mock object implements all the APIs defined in the service: ```go type myMockObject struct{} func (m *myMockObject) GetPrice(ctx context.Context, p *products.PriceParams) (*products.PriceResponse, error) { return &products.PriceResponse{Price: 100}, nil } func Test_Something(t *testing.T) { t.Parallel() // Run this test in parallel with other tests without the mock implementation interfering // This will cause a compile time error if myMockObject does not implement all the APIs defined in the products service et.MockService[products.Interface]("products", &myMockObject{}) } ``` ### Automatic generation of mock objects Thanks to the generated `Interface` interface, it's possible to automatically generate mock objects for your services using either [Mockery](https://vektra.github.io/mockery/latest/) or [GoMock](https://github.com/uber-go/mock). --- Source: https://encore.dev/docs/go/develop/testing Section: Building (Encore.go) # Automated testing > Confidence at speed Go comes with excellent built-in support for automated tests. Encore builds on top of this foundation, and lets you write tests in exactly the same way. We won't cover the basics of how to write tests here, see [the official Go docs](https://golang.org/pkg/testing/) for that. Let's instead focus on the difference between testing in Encore compared to a standard Go application. The main difference is that since Encore requires an extra compilation step, you must run your tests using `encore test` instead of `go test`. This is a wrapper that compiles the Encore app and then runs `go test`. It supports all the same flags that the `go test` command does. For example, use `encore test ./...` to run tests in all sub-directories, or just `encore test` for the current directory. ## Test tracing Encore comes with built-in test tracing for local development. You only need to open Encore's local development dashboard at [localhost:9400](http://localhost:9400) to see traces for all your tests. This makes it very simple to understand the root cause for why a test is failing. ![Test tracing](https://encore.dev/assets/docs/test_trace.png) ## Integration testing Since Encore removes almost all boilerplate, most of the code you write is business logic that involves databases and calling APIs between services. Such behavior is most easily tested with integration tests. When running tests, Encore automatically sets up the databases you need in a separate database cluster. They are additionally configured to skip `fsync` and to use an in-memory filesystem since durability is not a concern for automated tests. This drastically reduces the speed overhead of writing integration tests. In general, Encore applications tend to focus more on integration tests compared to traditional applications that are heavier on unit tests. This is nothing to worry about and is the recommended best practice. ### Temporary databases When Encore runs tests, by default it reuses the same database for all tests, to improve performance. However, this means that you need to take care when writing tests to ensure tests don't interfere with each other. If you instead want to have a separate database for a given test, you can use [`et.NewTestDatabase`](https://pkg.go.dev/encore.dev/et#NewTestDatabase) to create a temporary database that only exists for the duration of the test. The temporary test database is a fully-migrated database. It does not include any data written by other tests. > **Note** > > Under the hood, when you start running tests, Encore sets up a fresh "template database" and runs the database migrations > against that database. When you later call `et.NewTestDatabase`, Encore creates a new database by cloning the template database. ### Superuser database access Some test setup operations (such as `TRUNCATE TABLE` or other privileged DDL) require database superuser privileges that the application's regular database role doesn't have. To run such operations, use [`et.SQLDBWithSuperuser`](https://pkg.go.dev/encore.dev/et#SQLDBWithSuperuser) to obtain a copy of a database whose connections authenticate as the superuser role: ```go import "encore.dev/et" func TestSomething(t *testing.T) { superDB := et.SQLDBWithSuperuser(mydb) if _, err := superDB.Exec(ctx, "TRUNCATE TABLE users"); err != nil { t.Fatal(err) } // ... rest of the test } ``` ### Service Structs In tests, [service structs](https://encore.dev/docs/go/primitives/service-structs) are initialized on demand when the first API call is made to that service and then that instance of the service struct for all future tests. This means your tests can run faster as they don't have to each initialize all the service struct's each time a new test starts. However, in some situations you might be storing state in the service struct that would interfere with other tests. When you have a test you want to have its own instance of the service struct, you can use the `et.EnableServiceInstanceIsolation()` function within the test to enable this for just that test, while the rest of your tests will continue to use the shared instance. ## Test-only infrastructure Encore allows tests to define infrastructure resources specifically for testing. This can be useful for testing library code that interacts with infrastructure. For example, the [x.encore.dev/pubsub/outbox](https://pkg.go.dev/x.encore.dev/infra/pubsub/outbox) package defines a test-only database that is used to do integration testing of the outbox functionality. ## Testing from your IDE ### GoLand / IntelliJ Encore has an officially supported plugin [available in the JetBrains marketplace](https://plugins.jetbrains.com/plugin/20010-encore). It lets you run unit tests directly from within your IDE with support for debug mode and breakpoints. ### Visual Studio Code (VS Code) There's no official VS Code plugin available yet, but we are happy to include your contribution if you build one. Reach out on [Discord](https://encore.dev/discord) if you need help to get started. For advice on debugging when using VS Code, see the [Debugging docs](https://encore.dev/docs/go/how-to/debug). --- Source: https://encore.dev/docs/go/how-to/debug Section: Building (Encore.go) # Debug with Delve Encore makes it easy to debug your application using [Delve](https://github.com/go-delve/delve "Delve"). First, make sure you have `dlv` installed by running (Go 1.16 and later): ```shell $ go install github.com/go-delve/delve/cmd/dlv@latest ``` You have two debugger options, you can either debug by attaching to a running process or by starting the process in debug mode. ## Debug by starting the process in debug mode Run your Encore application with `encore run --debug=break`. This will launch your encore application with a headless Delve server, which will pause your application until a debugger is attached. ```shell $ encore run --debug=break API Base URL: http://localhost:4000 Dev Dashboard URL: http://localhost:9400/hello-world-cgu2 API server listening at: 127.0.0.1:2345 ``` Now it's time to attach the debugger. The instructions differ depending on how you would like to debug (in your terminal or in your editor). If instructions for your editor aren’t listed below, consult your editor for information on how to attach to a Delve server. ### Terminal debugging To debug in your terminal, run `dlv attach :2345`. You should see: ```shell $ dlv connect :2345 Type 'help' for list of commands. (dlv) ``` How to use Delve’s terminal interface for debugging is out of scope for this guide, but there are great resources available. For a good introduction, see [](https://golang.cafe/blog/golang-debugging-with-delve.html "Debugging with Delve"). ### Visual Studio Code To debug with VS Code you must first add a debug configuration. Press `Run -> Add Configuration`, choose `Go -> Connect to server`. Input `127.0.0.1` as host and `2345` as port. The resulting configuration should look something like this: ```json { "version": "0.2.0", "configurations": [ { "name": "Connect to server", "type": "go", "request": "attach", "mode": "remote", "remotePath": "${workspaceFolder}", "port": 2345, "host": "127.0.0.1" } ] } ``` Next, open the **Run and Debug** menu in the toolbar on the left, select `Connect to server` (the configuration you just created), and then press the green arrow. That’s it! You should be able to set breakpoints and have the Encore application pause when they’re hit like you would expect. ### Goland To debug with Goland, you must create a new Go Remote configuration. Press `Run | Edit Configurations`, click the `+` button, and choose `Go Remote`. Give it a name and hit `OK`. Now select the configuration you just created and press the green bug. That's it. You should be able to set breakpoints and have the Encore application pause when they’re hit like you would expect. ## Debug by attaching to a running process Run your Encore application with `encore run --debug`. This will cause Encore to print the Process ID to the terminal, which you will use to attach your debugger: ```shell $ encore run --debug API Base URL: http://localhost:4000 Dev Dashboard URL: http://localhost:9400/hello-world-cgu2 Process ID: 51894 1:48PM TRC registered endpoint path=/hello/:name service=hello endpoint=Hello ``` (Your process id will differ). When your Encore application is running, it’s time to attach the debugger. The instructions differ depending on how you would like to debug (in your terminal or in your editor). If instructions for your editor aren’t listed below, consult your editor for information on how to attach a debugger to a running process. ### Terminal debugging To debug in your terminal, run `dlv attach $PID` (replace `$PID` with your Process ID from the previous step). You should see: ```shell $ dlv attach 51894 Type 'help' for list of commands. (dlv) ``` How to use Delve’s terminal interface for debugging is out of scope for this guide, but there are great resources available. For a good introduction, see [](https://golang.cafe/blog/golang-debugging-with-delve.html "Debugging with Delve"). ### Visual Studio Code To debug with VS Code you must first add a debug configuration. Press `Run -> Add Configuration`, choose `Go -> Attach to local process`. In the generated configuration, you should see `"processId": 0` as a field. Replace `0` with the process id from above. Next, open the **Run and Debug** menu in the toolbar on the left, select Attach to Process (the configuration you just created), and then press the green arrow. That’s it! You should be able to set breakpoints and have the Encore application pause when they’re hit like you would expect. ### Goland To debug with Goland, you must first install the `gops` package. Open a terminal and run the following command ```shell go get -t github.com/google/gops/ ``` Then click `Run | Attach to Process`. If a notification window appears, click the `Invoke 'go get gops'` link. Once it has completed, click `Run | Attach to Process` again. In the dialog that appears, select the process with the process ID from above. That's it. You should be able to set breakpoints and have the Encore application pause when they’re hit like you would expect. --- Source: https://encore.dev/docs/go/observability/logging Section: Building (Encore.go) # Logging > Structured logging helps you understand your application Encore offers built-in support for structured logging, which combines a free-form log message with key-value pairs. Structured fields can be searched, filtered, grouped, and aggregated without parsing the message text. Logging is integrated with [distributed tracing](https://encore.dev/docs/tracing). A log message emitted while handling a request is recorded in that request's trace, alongside the API calls, database queries, and Pub/Sub operations that Encore captures on its own. Application logs add business context that Encore cannot infer from these operations. ## Usage Import `encore.dev/rlog` and call one of `Error`, `Warn`, `Info`, or `Debug`: ```go import "encore.dev/rlog" rlog.Info("order created", "order_id", "order_123", "customer_id", "customer_456", "total", 14900, "currency", "SEK", ) ``` The first parameter is the message. The remaining arguments alternate between string keys and values. Pass variable data as fields rather than interpolating it into the message: ```go // Prefer rlog.Info("order created", "order_id", orderID, "customer_id", customerID) // Avoid rlog.Info("order " + orderID + " created for customer " + customerID) ``` Interpolated values have to be parsed back out before they can be queried, and they make every message unique, so equivalent events can no longer be grouped or counted. ### Logging errors Pass errors as field values: ```go if err := chargeCustomer(ctx, customerID, amount); err != nil { rlog.Error("payment authorization failed", "err", err, "customer_id", customerID, "amount", amount, "currency", currency, ) return err } ``` Passing the original `error` value allows `rlog` to apply its error serialization. Converting it with `err.Error()` before calling the logger passes a plain string instead. ### Contextual loggers When several messages share the same fields, `rlog.With` returns a logging context that includes them in every subsequent event: ```go logger := rlog.With("order_id", orderID, "customer_id", customerID) logger.Info("order rejected", "reason", "payment_declined") logger.Info("order cancellation requested", "requested_by", "customer") ``` Use it for context that belongs to a domain operation and that Encore does not already know, such as an import ID or a tenant. Encore already attaches the service, endpoint, trace ID, and authenticated user ID to every log line; do not repeat these fields in a contextual logger. ## What to log - A business state transition, such as a subscription activating or an order shipping. - A decision the application made, together with the inputs that drove it. - A fallback or retry that allowed an operation to complete. - A failure, at the boundary that owns the recovery decision. - An interaction with an external system that needs domain context to interpret. - A security-relevant or administrative action. ```go rlog.Info("order rejected", "event", "order.rejected", "order_id", orderID, "reason", "insufficient_inventory", "product_id", productID, "requested_quantity", requestedQuantity, "available_quantity", availableQuantity, ) ``` ```go rlog.Warn("serving cached exchange rate", "event", "exchange_rate.fallback_used", "currency_pair", "EUR_SEK", "cache_age_seconds", cacheAgeSeconds, "provider_error_code", providerErrorCode, ) ``` ### What Encore already records Encore traces API requests, database queries, service-to-service calls, cache operations, and Pub/Sub activity, and the [local development dashboard](https://encore.dev/docs/dev-dash) renders them as a timeline. Logs that narrate the same execution flow duplicate the trace: ```go // All redundant with the trace rlog.Info("request started") rlog.Info("querying database") rlog.Info("calling payments service") rlog.Info("request completed") ``` Logging every successful query or function entry and exit also duplicates information already present in the trace. ## Logs, metrics, and traces - **Metrics** answer how often something happens. Use [metrics](https://encore.dev/docs/go/observability/metrics) for aggregate values. - **Traces** answer where the time went and what called what. - **Logs** record what happened, with the identifiers needed to investigate an individual event. Use a counter instead of emitting one `Info` log per event when only the number of events is needed, such as for cache hits. ## Log levels **`Error`** is for an operation that failed unexpectedly and could not complete: an unavailable dependency, data that violates an invariant, or a background job that has permanently failed. Expected outcomes such as a rejected login, a missing resource, failed validation, or a declined payment should not use `Error`. Logging expected outcomes at `Error` prevents the level from serving as a reliable alerting signal. **`Warn`** is for something unexpected that the application recovered from or continued through in a degraded state: falling back to stale data, ignoring malformed optional input, succeeding on retry, or approaching a soft limit. A condition that occurs continuously during normal operation is not a warning. **`Info`** is for meaningful, relatively low-volume business events: a user-visible state transition, a scheduled job completing, a configuration change, or a batch summary. **`Debug`** is for detail that helps an engineer investigate behavior and is not needed to operate the application. The minimum level is set by `log_level` in `encore.app` and applies to the whole application. The default minimum is `trace`, so all four `rlog` levels are emitted unless the level is configured. A `Debug` call in frequently executed code is therefore emitted in production by default. ## Message and field conventions Use the same message for equivalent events and preserve the variable data in fields. For event families that will be queried, counted, or consumed by automation, add a machine-readable `event` field alongside the human-readable message: ```go rlog.Info("payment authorized", "event", "payment.authorized", "payment_id", paymentID, "order_id", orderID, "amount", amount, "currency", currency, ) ``` Dashboards and alerts built on `event` continue to work if the message changes. Name events using `.`, such as `order.created`, `payment.failed`, and `subscription.cancelled`. Field names should mean the same thing everywhere, and a given field should keep its type across events. `"amount_minor_units", 14900` can be summed and compared; `"amount", "149.00 SEK"` has to be parsed first, and the parser breaks on the first entry written in another currency's format. Prefer enum-like values over prose, so use `"reason", "insufficient_inventory"` instead of a sentence describing the shortage. Encore's own fields use snake_case (`trace_id`, `service`, `endpoint`, `uid`). Use the same convention for application fields and one name for each concept. Field names beginning with `encore_` are reserved for internal use. `rlog` rewrites a field such as `encore_key` to `x_encore_key`. Use `encore.dev/rlog` for application events that need structured fields or trace integration. The standard library's `log` package and `fmt` output do not provide `rlog` fields or create log events in the active trace. ## Sensitive data Encore writes log fields without redacting them. Never log passwords or password hashes, API keys, access or refresh tokens, session identifiers, cookies, authorization headers, secret configuration values, or payment card and bank account details. Avoid logging whole request or response values, which may contain several of these at once. Prefer internal identifiers over personal values, such as `"user_id", userID` instead of old and new email addresses. When a value is needed for correlation but should not be stored, log a stable non-reversible fingerprint of it instead. Logs are durable production data. They are written to your cloud provider's log store, where access control, retention, exports, and backups govern who can read them. Fields are also recorded in the trace, so a sensitive value may be stored under two separate retention policies. Ordinary application logs are not an audit log. A formal audit trail usually needs explicit event schemas, defined retention, restricted access, tamper resistance, reliable delivery, and clear actor identity. For compliance-sensitive actions, write a dedicated audit event to durable storage; an application log alongside it is still useful for debugging. ## Log an error once Recording the same failure at every layer produces duplicate entries for one event: ```go func loadOrder(ctx context.Context, id string) (*Order, error) { order, err := queryOrder(ctx, id) if err != nil { rlog.Error("failed to load order", "err", err) return nil, err } return order, nil } func createInvoice(ctx context.Context, id string) error { _, err := loadOrder(ctx, id) if err != nil { rlog.Error("failed to create invoice", "err", err) // second entry, same failure return err } // ... return nil } ``` Log at the layer that owns the decision to retry, fall back, or abort, which is also the layer with the domain context to describe what failed. Lower layers return the error: ```go func loadOrder(ctx context.Context, id string) (*Order, error) { return queryOrder(ctx, id) } func createInvoice(ctx context.Context, orderID string) error { _, err := loadOrder(ctx, orderID) if err != nil { rlog.Error("invoice creation failed", "err", err, "order_id", orderID) return err } // ... return nil } ``` A lower layer logs when it suppresses the error, retries, detects a broken internal invariant, or holds information the returned error will not contain. Returning structured error information to the caller is often better than logging at the lower layer. ## Operation and batch summaries At the end of an operation, record its outcome together with relevant counts and its duration: ```go rlog.Info("account reconciliation completed", "event", "account.reconciliation_completed", "account_id", accountID, "transaction_count", transactionCount, "matched_count", matchedCount, "unmatched_count", unmatchedCount, "adjustments_created", adjustmentsCreated, "duration_ms", duration.Milliseconds(), ) ``` For a long-running operation, a start event distinguishes an operation that has not started from one that started but did not finish. The trace records the intermediate steps. A batch should record a summary. Logging each item in a 50,000-item batch produces 50,000 entries: ```go rlog.Info("user import completed", "event", "user_import.completed", "import_id", importID, "total_count", totalCount, "succeeded_count", succeededCount, "skipped_count", skippedCount, "failed_count", failedCount, "duration_ms", duration.Milliseconds(), ) ``` Individual failures can still be logged when they need investigation: ```go rlog.Warn("user import row rejected", "import_id", importID, "row_number", rowNumber, "reason", "invalid_email", ) ``` For very large batches, aggregate the failures by reason or log a bounded sample rather than every rejected record. ## Controlling log volume Log volume affects ingestion, storage, indexing, and query costs. High-volume entries in frequently executed code make relevant entries harder to find. Provider responses, database records, and configuration values may contain large fields, encoded data, or secrets. Log selected metadata instead of the complete value: ```go // Avoid rlog.Debug("provider response received", "response", response) // Prefer rlog.Debug("provider response received", "provider", "example-payments", "status_code", response.StatusCode, "request_id", response.Header.Get("X-Request-ID"), "item_count", len(response.Items), ) ``` For collections, log counts and boundary identifiers instead of contents: ```go rlog.Info("orders selected for reconciliation", "order_count", len(orders)) ``` Truncate long strings, cap slices at a few representative entries, and record a byte length instead of a document body: ```go const maxErrorBodyLength = 1000 previewLength := min(len(responseBody), maxErrorBodyLength) rlog.Warn("provider returned an invalid response", "provider", "example-payments", "status_code", statusCode, "response_body_length", len(responseBody), "response_body_preview", string(responseBody[:previewLength]), "response_body_truncated", len(responseBody) > maxErrorBodyLength, ) ``` Truncation limits volume but does not remove sensitive data. Review previews for secrets and personal data before logging them. Debug logs may be emitted in every environment and included in configured exports. Apply the same volume limits to `Debug` fields as to `Info` fields. Logs inside loops, Pub/Sub subscribers, and frequently called helpers can produce most of an application's log volume. Review volume by service, level, event name, and deployment version to find these call sites. For more information about the API, see the [`rlog` package documentation](https://pkg.go.dev/encore.dev/rlog). ## Live-streaming logs Stream logs from any environment directly to your terminal: ```shell $ encore logs --env=prod ``` --- Source: https://encore.dev/docs/go/observability/metrics Section: Building (Encore.go) # Metrics > Track custom metrics in your Go application Encore provides built-in support for defining custom metrics in your Go applications. Once defined, metrics are automatically collected and displayed in the Encore platform Dashboard, and can be exported to third-party observability services. See the [Platform metrics documentation](https://encore.dev/docs/platform/observability/metrics) for information about integrations with third-party services like Grafana Cloud and Datadog. ## Defining custom metrics Define custom metrics by importing the [`encore.dev/metrics`](https://pkg.go.dev/encore.dev/metrics) package and creating a new metric using one of the `metrics.NewCounter` or `metrics.NewGauge` functions. For example, to count the number of orders processed: ```go import "encore.dev/metrics" var OrdersProcessed = metrics.NewCounter[uint64]("orders_processed", metrics.CounterConfig{}) func process(order *Order) { // ... OrdersProcessed.Increment() } ``` ## Metric types Encore currently supports two metric types: counters and gauges. **Counters** measure the count of something. A counter's value must always increase, never decrease. (Note that the value gets reset to 0 when the application restarts.) Typical use cases include counting the number of requests, the amount of data processed, and so on. **Gauges** measure the current value of something. Unlike counters, a gauge's value can fluctuate up and down. Typical use cases include measuring CPU usage, the number of active instances running of a process, and so on. For information about their respective APIs, see the API documentation for [Counter](https://pkg.go.dev/encore.dev/metrics#Counter) and [Gauge](https://pkg.go.dev/encore.dev/metrics#Gauge). ### Counter example ```go import "encore.dev/metrics" var RequestsReceived = metrics.NewCounter[uint64]("requests_received", metrics.CounterConfig{}) func handleRequest() { RequestsReceived.Increment() // ... handle request } ``` ### Gauge example ```go import "encore.dev/metrics" var ActiveConnections = metrics.NewGauge[int64]("active_connections", metrics.GaugeConfig{}) func onConnect() { ActiveConnections.Add(1) } func onDisconnect() { ActiveConnections.Add(-1) } ``` ## Defining labels Encore's metrics package provides a type-safe way of attaching labels to metrics. To define labels, create a struct type representing the labels and then use `metrics.NewCounterGroup` or `metrics.NewGaugeGroup`. The Labels type must be a named struct, where each field corresponds to a single label. Each field must be of type `string`, `int`, or `bool`. ### Counter with labels ```go import "encore.dev/metrics" type Labels struct { Success bool } var OrdersProcessed = metrics.NewCounterGroup[Labels, uint64]("orders_processed", metrics.CounterConfig{}) func process(order *Order) { var success bool // ... populate success with true/false ... OrdersProcessed.With(Labels{Success: success}).Increment() } ``` ### Gauge with labels ```go import "encore.dev/metrics" type ConnectionLabels struct { Region string } var ActiveConnections = metrics.NewGaugeGroup[ConnectionLabels, int64]("active_connections", metrics.GaugeConfig{}) func onConnect(region string) { ActiveConnections.With(ConnectionLabels{Region: region}).Add(1) } ``` > **Important** > > Each combination of label values creates a unique time series tracked in memory and stored by the monitoring system. > Using numerous labels can lead to a combinatorial explosion, causing high cloud expenses and degraded performance. > > As a general rule, limit the unique time series to tens or hundreds at most, rather than thousands. --- Source: https://encore.dev/docs/go/develop/config Section: Building (Encore.go) # Configuration > Define behavior in specific environments Configuration files let you define default behavior for your application, and override it for specific environments. This allows you to make changes without affecting deployments in other environments. Encore supports configuration files written in [CUE](https://cuelang.org/), which is a superset of JSON. It adds the following: - C-style comments - Quotes may be omitted from field names without special characters - Commas at the end of fields are optional - A comma after last element in list is allowed - The outer curly braces on the file are optional - [Expressions](https://cuelang.org/docs/tutorials/tour/expressions/) such as interpolation, comprehensions and conditionals are supported. > **Important** > > For sensitive data use Encore's [secrets management](https://encore.dev/docs/go/primitives/secrets) functionality instead of configuration. ## Using Config Inside your service, you can call `config.Load[*SomeConfigType]()` to load the config. This must be done at the package level, and not inside a function. See more in the [package documentation](https://pkg.go.dev/encore.dev/config#Load). Here's an example implementation: ```go package mysvc import ( "encore.dev/config" ) type SomeConfigType struct { ReadOnly config.Bool // Put the system into read-only mode Example config.String } var cfg *SomeConfigType = config.Load[*SomeConfigType]() ``` The type you pass as a type parameter to this function will be used to generate a `encore.gen.cue` file in your services directory. This file will contain both the CUE definition for your configuration type, and some [metadata](#provided-meta-values) that Encore will provide to your service at runtime. This allows you to change the final value of your configuration based on the environment the application is running in. Any files ending with `.cue` in your service directory or sub-directories will be loaded by Encore and given to CUE to unify and compute a final configuration. **Example CUE files** ``` -- mysvc/encore.gen.cue -- // Code generated by encore. DO NOT EDIT. package mysvc #Meta: { APIBaseURL: string Environment: { Name: string Type: "production" | "development" | "ephemeral" | "test" Cloud: "aws" | "gcp" | "encore" | "local" } } #Config: { ReadOnly: bool // Put the system into read-only mode Example: string } #Config -- mysvc/myconfig.cue -- // Set example to "hello world" Example: "hello world" // By default we're not in read only mode ReadOnly: bool | *false // But on the old production environment, we're in read only mode if #Meta.Environment.Name == "old-prod" { ReadOnly: true } ``` > **Note** > > Loading configuration > is only supported in services and the loaded data can not be referenced from packages outside that service. ### CUE tags in Go Structs You can use the `cue` tag in your Go to specify additional constraints on your configuration. For example: ```go type FooBar { A int `cue:">100"` B int `cue:"A-50"` // If A is set, B can be inferred by CUE C int `cue:"A+B"` // Which then allows CUE to infer this too } var _ = config.Load[*FooBar]() ``` Will result in the following CUE type definition being generated: ```cue #Config: { A: int & >100 B: int & A-50 // If A is set, B can be inferred by CUE C: int & A+B // Which then allows CUE to infer this too } ``` ## Config Wrappers Encore provides type wrappers for config in the form of `config.Value[T]` and `config.Values[T]` which expand into functions of type `T` and `[]T` respectively. These functions allow you to override the default value of your configuration in your CUE files inside tests, where only code run from that test will see the override. In the future we plan to support real-time updating of configuration values on running applications, thus using these wrappers in your configuration today will future proof your code and allow you to automatically take advantage of this feature when it is available. Any type supported in API requests and responses can be used as the type for a config wrapper. However for convenience, Encore ships with the following inbuilt aliases for the config wrappers: - `config.String`, `config.Bool`, `config.Int`, `config.Uint`, `config.Int8`, `config.Int16`, `config.Int32`, `config.In64`, `config.Uint8`, `config.Uint16`, `config.Uint32`, `config.Uint64`, `config.Float32`, `config.Float64`, `config.Bytes`, `config.Time`, `config.UUID` **Example Application using Wrappers** ```go -- svc/svc.go -- type mysvc import ( "encore.dev/config" ) type Server struct { // The config wrappers do not have to be in the top level struct Enabled config.Bool Port config.Int } type SvcConfig struct { GameServerPorts config.Values[Server] } var cfg = config.Load[*SvcConfig]() func startServers() { for _, server := range cfg.GameServerPorts() { if server.Enabled() { go startServer(server.Port()) } } } func startServer(port int) { // ... } -- svc/servers.cue -- GameServerPorts: [ { Enabled: false Port: 12345 }, { Enabled: true Port: 1337 }, ] ``` ## Provided Meta Values When your application is running, Encore will provide information about that environment to your CUE files, which you can use to filter on. These fields can be found in the `encore.gen.cue` file which Encore will generate when you add a call to load config. Encore provides the following meta values: - **APIBaseURL**: The base URL of the Encore API, which can be used to make API calls to the application. - **Environment**: A struct containing information about the environment the application is running in.
   **Name**: The name of the environment
   **Type**: One of `production`, `development`, `ephemeral` or `test`.
   **Cloud**: The cloud the app is running on, which is one of `aws`, `gcp`, `encore` or `local`.
The following are useful conditionals you can use in your CUE files: ```cue // An application running due to `encore run` if #Meta.Environment.Type == "development" && #Meta.Environment.Cloud == "local" {} // An application running in a development environment in the Cloud if #Meta.Environment.Type == "development" && #Meta.Environment.Cloud != "local" {} // An application running in a production environment if #Meta.Environment.Type == "production" {} // An application running in an environment that Encore has created // for an open Pull Request on Github if #Meta.Environment.Type == "ephemeral" {} ``` ## Testing with Config Through the provided meta values, your applications configuration can have different values in tests, compared to when the application is running. This can be useful to prevent external side effects from your tests, such as emailing customers across all test. Sometimes however, you may want to test specific behaviors based on different configurations (such as disabling user signups), in this scenario using the Meta data does not give you fine enough control. To allow you to set a configuration value at a per test level, Encore provides the helper function [`et.SetCfg`](https://pkg.go.dev/encore.dev/et#SetCfg). You can use this function to set a new value only in the current test and any sub tests, while all other tests will continue to use the value defined in the CUE files. ```go -- config.cue -- // By default we want to send emails SendEmails: bool | *true // But in all tests we want to disable emails if #Meta.Environment.Type == "test" { SendEmails: false } -- signup.go -- import ( "context" "encore.dev/config" ) type Config struct { SendEmails config.Bool } var cfg = config.Load[Config]() //encore:api public func Signup(ctx context.Context, p *SignupParams) error { user := createUser(p) if cfg.SendEmails() { SendWelcomeEmail(user) } return nil } -- signup_test.go -- import ( "errors" "testing" "encore.dev/et" ) func TestSignup(t *testing.T) { err := Signup(context.Background(), &SignupParams { ... }) if err != nil { // We don't expect an error here t.Fatal(err) } if emailWasSent() { // We don't expect an email to be sent // as it's disabled for all tests t.Fatal("email was sent") } } func TestSignup_TestEmails(t *testing.T) { // For this test, we want to enable the welcome // emails so we can test that they are sent et.SetCfg(cfg.SendEmails, true) err := Signup(context.Background(), &SignupParams { ... }) if err != nil { // We don't expect an error here t.Fatal(err) } // Check the email was sent if !emailWasSent() { t.Fatal("email was not sent") } } ``` ## Useful CUE Patterns If you're new to CUE, we'd recommend checking out the [CUE documentation](https://cuelang.org/docs/) and [cuetorials](https://cuetorials.com/), however to get you started, here are some useful patterns you can use in your CUE files. ### Defaults CUE supports the concept of a default value, which it will use if no other concrete value is provided. This can be useful for when you normally want one value, but occasionally might want to provide an override in a certain scenario. A default value is specified by prefixing it with a `*`. ```cue // ReadOnlyMode is a boolean and if we don't provide a value, it // will default to false. ReadOnlyMode: bool | *false if #Meta.Environment.Name == "old-prod" { // On this environment, we want to set ReadOnlyMode to true ReadOnlyMode: true } ``` ### Validation within CUE Any field prefixed with an `_` will not be exported to the concrete configuration once evaluated by CUE and can be used to hold intermediate values. Because CUE allows you to define the same field as many times as you want, as long as the values unify, we can build complex validation logic. ```cue import ( "list" // import CUE's list package ) // Set some port numbers defaulting just to 8080 // but in development including 8443 portNumbers: [...int] | *[8080] if #Meta.Environment.Type == "development" { portNumbers: [8080, 8443] } // Port numbers must be an array and all values // are integers 1024 or above. portNumbers: [...int & >= 1024] // The ports are considered valid if they contain the port number 8080. _portsAreValid: list.Contains(portNumbers, 8080) // Ensure that the ports are valid by constraining the value to be true. // CUE will report an error if the value is false (that is if the portNumbers list // does not contain the value 8080). _portsAreValid: true ``` ### Switch Statements If statements in CUE do not have else branches, which can make it difficult to write complex conditionals, we however can use an array to emulate a switch statement, where the first value that matches the condition is returned. The following example will set `SendEmailsFrom` to a single string. ```cue SendEmailsFrom: [ // These act as individual case statements if #Meta.Environment.Type == "production" { "noreply@example.com" }, if #Meta.Environment.Name == "staging" { "staging@example.com" }, // This last value without a condition acts as the default case "dev-system@example.dev", ][0] // Return the first value which matches the condition ``` ### Using Map Keys as Values CUE allows us to extract map keys and use them as values to simplify the config we need to write and minimize duplication. ```cue // Define the type we want to use #Server: { server: string port: int & > 1024 enabled: bool | *true } // Specify that servers is a map of strings to #Server // where they key we assign the variable Name servers: [Name=string]: #Server & { // Then we union the key with the value of server server: Name } servers: { "Foo": { port: 8080 }, "Bar": { port: 8081 enabled: false }, } ``` This will result in the concrete configuration of: ```json { "servers": { "Foo": { "server": "Foo", "port": 8080, "enabled": true }, "Bar": { "server": "Bar", "port": 8081, "enabled": false } } } ``` --- Source: https://encore.dev/docs/go/primitives/defining-apis Section: Building (Encore.go) # Defining Type-Safe APIs > Simplifying type-safe API development Encore.go enables you to create type-safe APIs from regular Go functions. To define an API, add the `//encore:api` annotation to a function in your code. This tells Encore that the function is an API endpoint and Encore will automatically generate the necessary boilerplate at compile-time. In the example below, we define the API endpoint `Ping`, in the `hello` service, which gets exposed as `hello.Ping`. ```go package hello // service name //encore:api public func Ping(ctx context.Context, params *PingParams) (*PingResponse, error) { msg := fmt.Sprintf("Hello, %s!", params.Name) return &PingResponse{Message: msg}, nil } ``` **Related example:** [encoredev/examples/hello-world](https://github.com/encoredev/examples/tree/main/hello-world) Hello World REST API example application. Create it locally with `encore app create --example=hello-world`. ## Access controls When you define an API, you have three options for how it can be accessed: * `//encore:api public` – defines a public API that anybody on the internet can call. * `//encore:api private` – defines a private API that is never accessible to the outside world. It can only be called from other services in your app and via cron jobs. * `//encore:api auth` – defines a public API that anybody can call, but requires valid authentication. You can optionally send in auth data to `public` and `private` APIs, in which case the auth handler will be used. When used for `private` APIs, they are still not accessible from the outside world. For more on defining APIs that require authentication, see the [authentication guide](https://encore.dev/docs/go/develop/auth). ## API Schemas ### Request and response schemas In the example above we defined an API that uses request and response schemas. The request data is of type `PingParams` and the response data of type `PingResponse`. That means we need to define them like so: ```go package hello // service name // PingParams is the request data for the Ping endpoint. type PingParams struct { Name string } // PingResponse is the response data for the Ping endpoint. type PingResponse struct { Message string } // Ping is an API endpoint that responds with a simple response. // This is exposed as "hello.Ping". //encore:api public func Ping(ctx context.Context, params *PingParams) (*PingResponse, error) { msg := fmt.Sprintf("Hello, %s!", params.Name) return &PingResponse{Message: msg}, nil } ``` Request and response schemas are both optional. There are four different ways of defining an API: **Using both request and response data:**
`func Foo(ctx context.Context, p *Params) (*Response, error)` **Only returning a response:**
`func Foo(ctx context.Context) (*Response, error)` **With only request data:**
`func Foo(ctx context.Context, p *Params) error` **Without any request or response data:**
`func Foo(ctx context.Context) error` As you can see, two parts are always present: the `ctx context.Context` parameter and the `error` return value. The `ctx` parameter is used for *cancellation*. It lets you detect when the caller is no longer interested in the result, and lets you abort the request processing and save resources that nobody needs. [Learn more about contexts on the Go blog](https://blog.golang.org/context). The `error` return type is always required because APIs can always fail from the caller's perspective. Therefore even though our simple `Ping` API endpoint above never fails in its implementation, from the perspective of the caller perhaps the service is crashing or the network is down and the service cannot be reached. This approach is simple but very powerful. It lets Encore use [static analysis](https://encore.dev/docs/understanding-encore) to understand the request and response schemas of all your APIs, which enables Encore to automatically generate API documentation, type-safe API clients, and much more. ### Request and response data types Request and response data types are structs (or pointers to structs) with optional field tags, which Encore uses to encode API requests to HTTP messages. The same struct can be used for requests and responses, but the `query` tag is ignored when generating responses. All tags except `json` are ignored for nested tags, which means you can only define `header` and `query` parameters for root level fields. For example, this struct: ```go type NestedRequestResponse struct { Header string `header:"X-Header"`// this field will be read from the http header Query string `query:"query"`// this field will be read from the query string Body1 string `json:"body1"` Nested struct { Header2 string `header:"X-Header2"`// this field will be read from the body Query2 string `query:"query2"`// this field will be read from the body Body2 string `json:"body2"` } `json:"nested"` } ``` Would be unmarshalled from this request: ```output POST /example?query=a%20query HTTP/1.1 Content-Type: application/json X-Header: A header { "body1": "a body", "nested": { "Header2": "not a header", "Query2": "not a query", "body2": "a nested body" } } ``` And marshalled to this response: ```output HTTP/1.1 200 OK Content-Type: application/json X-Header: A header { "Query": "not a query", "body1": "a body", "nested": { "Header2": "not a header", "Query2": "not a query", "body2": "a nested body" } } ``` ### Path parameters Path parameters are specified by the `path` field in the `//encore:api` annotation. To specify a placeholder variable, use `:name` and add a function parameter with the same name to the function signature. Encore parses the incoming request URL and makes sure it matches the type of the parameter. The last segment of the path can be parsed as a wildcard parameter by using `*name` with a matching function parameter. ```go // GetBlogPost retrieves a blog post by id. //encore:api public method=GET path=/blog/:id/*path func GetBlogPost(ctx context.Context, id int, path string) (*BlogPost, error) { // Use id to query database... } ``` ### Fallback routes Encore supports defining fallback routes that will be called if no other endpoint matches the request, using the syntax `path=/!fallback`. This is often useful when migrating an existing backend service over to Encore, as it allows you to gradually migrate endpoints over to Encore while routing the remaining endpoints to the existing HTTP router using a raw endpoint with a fallback route. For example: ```go //encore:service type Service struct { oldRouter *gin.Engine // existing HTTP router } // Route all requests to the existing HTTP router if no other endpoint matches. //encore:api public raw path=/!fallback func (s *Service) Fallback(w http.ResponseWriter, req *http.Request) { s.oldRouter.ServeHTTP(w, req) } ``` ### Headers Headers are defined by the `header` field tag, which can be used in both request and response data types. The tag name is used to translate between the struct field and http headers. In the example below, the `Language` field of `ListBlogPost` will be fetched from the `Accept-Language` HTTP header. ```go type ListBlogPost struct { Language string `header:"Accept-Language"` Author string // Not a header } ``` Headers also support slice types (e.g. `[]string`), which map to multiple HTTP header values. > **Note** > > Generated TypeScript/JavaScript clients use the Fetch API which [combines multiple header values](https://fetch.spec.whatwg.org/#concept-header-list-combine) into a single comma-joined string. This means header arrays will contain a single element with the joined value. The exception is `Set-Cookie`, which the Fetch API preserves as separate values. ### Cookies Cookies can be set in the response by using the `header` tag with the `Set-Cookie` header name. ```go type LoginResponse struct { SessionID string `header:"Set-Cookie"` } //encore:api public method=POST path=/login func Login(ctx context.Context) (*LoginResponse, error) { return &LoginResponse{SessionID: "session=123"}, nil } ```` The cookies can then be read using e.g. [structured auth data](https://encore.dev/docs/go/develop/auth#accepting-structured-auth-information). ### Query parameters For `GET`, `HEAD` and `DELETE` requests, parameters are read from the query string by default. The query parameter name defaults to the [snake-case](https://en.wikipedia.org/wiki/Snake_case) encoded name of the corresponding struct field (e.g. BlogPost becomes blog_post). The `query` field tag can be used to parse a field from the query string for other HTTP methods (e.g. POST) and to override the default parameter name. Query strings are not supported in HTTP responses and therefore `query` tags in response types are ignored. In the example below, the `PageLimit` field will be read from the `limit` query parameter, whereas the `Author` field will be parsed from the query string (as `author`) only if the method of the request is `GET`, `HEAD` or `DELETE`. ```go type ListBlogPost struct { PageLimit int `query:"limit"` // always a query parameter Author string // query if GET, HEAD or DELETE, otherwise body parameter } ``` When fetching data with `GET` endpoints, it's common to receive additional parameters for optional behavior, like filtering a list or changing the sort order. When you use a struct type as the last argument in the function signature, Encore automatically parses these fields from the HTTP query string (for the `GET`, `HEAD`, and `DELETE` methods). For example, if you want to have a `ListBlogPosts` endpoint: ```go type ListParams struct { Limit uint // number of blog posts to return Offset uint // number of blog posts to skip, for pagination } type ListResponse struct { Posts []*BlogPost } //encore:api public method=GET path=/blog func ListBlogPosts(ctx context.Context, opts *ListParams) (*ListResponse, error) { // Use limit and offset to query database... } ``` This could then be queried as `/blog?limit=10&offset=20`. Query parameters are more limited than structured JSON data, and can only consist of basic types (`string`, `bool`, integer and floating point numbers), time.Time, [Encore's UUID types](https://pkg.go.dev/encore.dev/types/uuid#UUID) and slices of those types. time.Time values in path, query and header parameters are encoded as RFC 3339, including fractional seconds when present (e.g. 2026-10-28T09:30:00.123456Z). ### Body parameters Encore will default to reading request parameters from the body (as JSON) for all HTTP methods except `GET`, `HEAD` or `DELETE`. The name of the body parameter defaults to the field name, but can be overridden by the `json` tag. Response fields will be serialized as JSON in the HTTP body unless the `header` tag is set. There is no tag to force a field to be read from the body, as some infrastructure entities do not support body content in `GET`, `HEAD` or `DELETE` requests. ```go type CreateBlogPost struct { Subject string `json:"limit"` // query if GET, HEAD or DELETE, otherwise body parameter Author string // query if GET, HEAD or DELETE, otherwise body parameter } ``` ### Optional types Encore supports optional types using the `option.Option[T]` type from the `encore.dev/types/option` package. This can be used in request and response schemas to indicate that the value is not always set. See the [package documentation](https://pkg.go.dev/encore.dev/types/option) for more information on usage. ### Supported types The table below lists the data types supported by each HTTP message location. | Type | Header | Path | Query | Body | | ---------------- | ------ | ---- | ----- | ---- | | bool | X | X | X | X | | numeric | X | X | X | X | | string | X | X | X | X | | time.Time | X | X | X | X | | uuid.UUID | X | X | X | X | | json.RawMessage | X | X | X | X | | option.Option[T] | X | | X | X | | pointer | X | | X | X | | list | X | | X | X | | struct | | | | X | | map | | | | X | ## Sensitive data Encore.go comes with built-in tracing functionality that automatically captures request and response payloads to simplify debugging. While helpful, that's not always desirable. For instance when a request or response payload contains sensitive data, such as API keys or personally identifiable information (PII). For those use cases Encore supports marking a field as sensitive using the struct tag `encore:"sensitive"`. Encore's tracing system will automatically redact fields tagged as sensitive. This works for both individual values as well as nested fields. Note that inputs to [auth handlers](https://encore.dev/docs/go/develop/auth) are automatically marked as sensitive and are always redacted. Raw endpoints lack a schema, which means there's no way to add a struct tag to mark certain data as sensitive. For this reason Encore supports tagging the whole API endpoint as sensitive by adding `sensitive` to the `//encore:api` annotation. This will cause the whole request and response payload to be redacted, including all request and response headers. > **Note** > > The `encore:"sensitive"` tag is ignored for local development environments to make development and debugging with the Local Development Dashboard easier. ### Example ```go package blog // service name import ( "time" "encore.dev/types/uuid" ) type Updates struct { Author string `json:"author,omitempty"` PublishTime time.Time `json:"publish_time,omitempty"` } // BatchUpdateParams is the request data for the BatchUpdate endpoint. type BatchUpdateParams struct { Requester string `header:"X-Requester"` RequestTime time.Time `header:"X-Request-Time"` CurrentAuthor string `query:"author"` Updates *Updates `json:"updates"` MySecretKey string `encore:"sensitive"` } // BatchUpdateResponse is the response data for the BatchUpdate endpoint. type BatchUpdateResponse struct { ServedBy string `header:"X-Served-By"` UpdatedIDs []uuid.UUID `json:"updated_ids"` } //encore:api public method=POST path=/section/:sectionID/posts func BatchUpdate(ctx context.Context, sectionID string, params *BatchUpdateParams) (*BatchUpdateResponse, error) { // Update blog posts for section return &BatchUpdateResponse{ServedBy: hostname, UpdatedIDs: ids}, nil } ``` ## REST APIs Encore has support for RESTful APIs and lets you easily define resource-oriented API URLs, parse parameters out of them, and more. To create a REST API, start by defining an endpoint and specify the `method` and `path` fields in the `//encore:api` comment. To specify a placeholder variable, use `:name` and add a function parameter with the same name to the function signature. Encore parses the incoming request URL and makes sure it matches the type of the parameter. For example, if you want to have a `GetBlogPost` endpoint that takes a numeric id as a parameter: ```go // GetBlogPost retrieves a blog post by id. //encore:api public method=GET path=/blog/:id func GetBlogPost(ctx context.Context, id int) (*BlogPost, error) { // Use id to query database... } ``` You can also combine path parameters with body payloads. For example, if you want to have an `UpdateBlogPost` endpoint: ```go // UpdateBlogPost updates an existing blog post by id. //encore:api public method=PUT path=/blog/:id func UpdateBlogPost(ctx context.Context, id int, post *BlogPost) error { // Use `post` to update the blog post with the given id. } ``` > **Important** > > You cannot define paths that conflict with each other, including paths > where the static part can be mistaken for a parameter, e.g both `/blog` and `/blog/:id` would conflict with `/:username`. As a rule of thumb, try to place path parameters at the end of the path and prefix them with the service name, e.g: ``` GET /blog/posts GET /blog/posts/:id GET /user/profile/:username GET /user/me ``` ## Custom HTTP status codes By default, Encore automatically sets appropriate HTTP status codes for your API responses. We recommend using these default status codes, but there are situations where you might need to set a custom HTTP status code, such as when porting an existing API that clients depend on for specific status codes. To set a custom HTTP status code, use the `encore:"httpstatus"` struct tag on a field in your response type: ```go type Response struct { Message string `json:"message"` Status int `encore:"httpstatus"` } //encore:api public method=GET path=/example func Example(ctx context.Context) (*Response, error) { return &Response{ Message: "Hello", Status: 201, // HTTP 201 Created }, nil } ``` The field with the `encore:"httpstatus"` tag can be an integer type and should contain a valid HTTP status code value. --- Source: https://encore.dev/docs/go/primitives/api-calls Section: Building (Encore.go) # API Calls > Making API calls is as simple as making function calls Calling an API endpoint looks like a regular function call with Encore.go. To call an endpoint you first import the other service as a Go package using `import "encore.app/package-name"` and then call the API endpoint like a regular function. Encore will automatically generate the necessary boilerplate at compile-time. In the example below, we import the service package `hello` and call the `Ping` endpoint using a function call to `hello.Ping`. ```go import "encore.app/hello" // import service //encore:api public func MyOtherAPI(ctx context.Context) error { resp, err := hello.Ping(ctx, &hello.PingParams{Name: "World"}) if err == nil { log.Println(resp.Message) // "Hello, World!" } return err } ``` **Related example:** [encoredev/examples/trello-clone](https://github.com/encoredev/examples/tree/main/trello-clone) Simple microservices example application with service-to-service API calls. Create it locally with `encore app create --example=trello-clone`. This means your development workflow is as simple as building a monolith, even if you use multiple services. You also get all the benefits of function calls, like compile-time checking of all the parameters and auto-completion in your editor, while still allowing the division of code into logical components, services, and systems. Then when building your application, Encore uses [static analysis](https://encore.dev/docs/understanding-encore) to parse all API calls and compiles them to proper API calls. ## Current Request By using Encore's [current request API](https://pkg.go.dev/encore.dev/#Request) you can get meta-information about the current request. Including the type of request, the time the request started, the service and endpoint called and the path which was called on the service. For more information, see the [metadata documentation](https://encore.dev/docs/go/develop/metadata). --- Source: https://encore.dev/docs/go/primitives/raw-endpoints Section: Building (Encore.go) # Raw Endpoints > Drop down in abstraction to access the raw HTTP request Sometimes you need to operate a lower abstraction than Encore.go normally provides. For example, you might want to access the underlying HTTP request, often useful for things like accepting webhooks. Encore.go has you covered using "raw endpoints". To define a raw endpoint, change the `//encore:api` annotation and function signature like so: ```go package service import "net/http" // Webhook receives incoming webhooks from Some Service That Sends Webhooks. //encore:api public raw func Webhook(w http.ResponseWriter, req *http.Request) { // ... operate on the raw HTTP request ... } ``` Like any other Encore API endpoint, once deployed this will be exposed at the URL:
`https://-.encr.app/service.Webhook`. Just like regular endpoints, raw endpoints support the use of `:id` and `*wildcard` segments. Experienced Go developers will have already noted this is just a regular Go HTTP handler. (See the net/http documentation for how Go HTTP handlers work.) Learn more about receiving webhooks and using WebSockets in the [receiving regular HTTP requests guide](https://encore.dev/docs/go/how-to/http-requests). **Related example:** [encoredev/examples/slack-bot](https://github.com/encoredev/examples/tree/main/slack-bot) Slack Bot example application that uses Raw endpoints to accept webhooks. Create it locally with `encore app create --example=slack-bot`. --- Source: https://encore.dev/docs/go/primitives/api-errors Section: Building (Encore.go) # API Errors > Returning structured error information from your APIs Encore supports returning structured error information from your APIs using the [encore.dev/beta/errs](https://pkg.go.dev/encore.dev/beta/errs) package. Errors are propagated across the network to the [generated clients](https://encore.dev/docs/go/cli/client-generation) and can be used within your front-ends without having to build any custom marshalling code. ## The errs.Error type Structured errors are represented by the `errs.Error` type: ```go type Error struct { // Code is the error code to return. Code ErrCode `json:"code"` // Message is a descriptive message of the error. Message string `json:"message"` // Details are user-defined additional details. Details ErrDetails `json:"details"` // Meta are arbitrary key-value pairs for use within // the Encore application. They are not exposed to external clients. Meta Metadata `json:"-"` } ``` Returning an `*errs.Error` from an Encore API endpoint will result in Encore serializing this struct to JSON and returning it in the response. Additionally Encore will set the HTTP status code to match the error code (see the mapping table below). For example: ```go return &errs.Error{ Code: errs.NotFound, Message: "sprocket not found", } ``` Causes Encore to respond with a `HTTP 404` error with body: ```json { "code": "not_found", "message": "sprocket not found", "details": null } ``` ## Error Wrapping Encore applications are encouraged to always use the `errs` package to manipulate errors. It supports wrapping errors to gradually add more error information, and lets you easily define both structured error details to return to external clients, as well as internal key-value metadata for debugging and error handling. ```go func Wrap(err error, msg string, metaPairs ...interface{}) error ``` Use `errs.Wrap` to conveniently wrap an error, adding additional context and converting it to an `*errs.Error`. If `err` is nil it returns `nil`. If `err` is already an `*errs.Error` it copies the Code, Details, and Meta fields over. The variadic `metaPairs` parameter must be key-value pairs, where the key is always a `string` and the value can be any built-in type. Existing key-value pairs from the `err` are merged into the new `*Error`. ```go func WrapCode(err error, code ErrCode, msg string, metaPairs ...interface{}) error ``` `errs.WrapCode` is like `errs.Wrap` but also sets the error code. ```go func Convert(err error) error ``` `errs.Convert` converts an error to an `*errs.Error`. If the error is already an `*errs.Error` it returns it unmodified. If `err` is nil it returns nil. ## Error Codes The `errs` package defines error codes for common error scenarios. They are identical to the codes defined by `gRPC` for interoperability. The table below summarizes the error codes. You can find additional documentation about when to use them in the [package documentation](https://pkg.go.dev/encore.dev/beta/errs#ErrCode). | Code | String | HTTP Status | | -------------------- | ----------------------- | ------------------------- | | `OK` | `"ok"` | 200 OK | | `Canceled` | `"canceled"` | 499 Client Closed Request | | `Unknown` | `"unknown"` | 500 Internal Server Error | | `InvalidArgument` | `"invalid_argument"` | 400 Bad Request | | `DeadlineExceeded` | `"deadline_exceeded"` | 504 Gateway Timeout | | `NotFound` | `"not_found"` | 404 Not Found | | `AlreadyExists` | `"already_exists"` | 409 Conflict | | `PermissionDenied` | `"permission_denied"` | 403 Forbidden | | `ResourceExhausted` | `"resource_exhausted"` | 429 Too Many Requests | | `FailedPrecondition` | `"failed_precondition"` | 400 Bad Request | | `Aborted` | `"aborted"` | 409 Conflict | | `OutOfRange` | `"out_of_range"` | 400 Bad Request | | `Unimplemented` | `"unimplemented"` | 501 Not Implemented | | `Internal` | `"internal"` | 500 Internal Server Error | | `Unavailable` | `"unavailable"` | 503 Unavailable | | `DataLoss` | `"data_loss"` | 500 Internal Server Error | | `Unauthenticated` | `"unauthenticated"` | 401 Unauthorized | ## Error Building In cases where you have complex business logic, or multiple error returns, it's convenient to gradually add metadata to your error. For this purpose Encore provides `errs.Builder`. The builder lets you gradually set aspects of the error, using a chaining API design. Use `errs.B()` to get a new builder that you can start chaining with directly. When you want to return the constructed error call the `.Err() `method. For example: ```go func getBoard(ctx context.Context, boardID int64) (*Board, error) { // Construct a new error builder with errs.B() eb := errs.B().Meta("board_id", boardID) b := &Board{ID: boardID} err := db.QueryRow(ctx, ` SELECT name, created FROM board WHERE id = $1 `, boardID).Scan(&b.Name, &b.Created) if errors.Is(err, sqldb.ErrNoRows) { // Return a "board not found" error with code == NotFound return nil, eb.Code(errs.NotFound).Msg("board not found").Err() } else if err != nil { // Return a general error return nil, eb.Cause(err).Msg("could not get board").Err() } // ... } ``` ## Inspecting API Errors When you call another API within Encore, the returned errors are always wrapped in `*errs.Error`. You can inspect the error information either by casting to `*errs.Error`, or using the below helper methods. ```go func Code(err error) ErrCode ``` `errs.Code` returns the error code. If the error was not an `*errs.Error` it returns `errs.Unknown`. ```go func Meta(err error) Metadata type Metadata map[string]interface{} ``` `errs.Meta` returns any structured metadata present in the error. If the error was not an `*errs.Error` it returns nil. Unlike when you return error information to external clients, all the metadata is sent to the calling service, making debugging even easier. ```go func Details(err error) ErrDetails ``` `errs.Details` returns the structured error details. If the error was not an `*errs.Error` or the error lacked details, it returns nil. --- Source: https://encore.dev/docs/go/develop/validation Section: Building (Encore.go) # Validation > Making sure everything's right in the world When receiving incoming requests it's best practice to validate the payload to make sure it meets your expectations, contains all the necessary fields, and so on. Encore provides an out-of-the-box middleware that automatically validates incoming requests if the request type implements the method `Validate() error`. If it does, Encore will call this method after deserializing the request payload, and only call your API handler (and other middleware) if the validation function returns `nil`. If the validation function returns an [`*errs.Error`](https://encore.dev/docs/go/primitives/api-errors) that error is reported unmodified to the caller. Other errors are converted to an `*errs.Error` with code `InvalidArgument`, which results in a HTTP response with status code `400 Bad Request`. This design means that it's easy to use your validation library of choice. In the future we're looking to provide an out-of-the-box validation library for an even better developer experience. --- Source: https://encore.dev/docs/go/primitives/service-structs Section: Building (Encore.go) # Service structs Encore lets you define a type, called a service struct, to represent your running service. This lets you define an initialization function (similar to the `main` function in regular Go programs). You can also define API endpoints as methods on the service struct type, enabling you to use [dependency injection](https://encore.dev/docs/go/how-to/dependency-injection) for testing purposes. It works by defining a struct type of your choice (typically called `Service`) and declaring it with `//encore:service`. Then, you can define a special function named `initService` (or `initWhatever` if you named the type `Whatever`) that gets called by Encore to initialize your service when it starts up. It looks like this: ```go //encore:service type Service struct { // Add your dependencies here } func initService() (*Service, error) { // Write your service initialization code here. } //encore:api public func (s *Service) MyAPI(ctx context.Context) error { // ... } ``` **Related example:** [encoredev/examples/uptime](https://github.com/encoredev/examples/tree/main/uptime) Event-driven example application using service structs. Create it locally with `encore app create --example=uptime`. ## Calling APIs defined on service structs When using a service struct like above, Encore will create a file named `encore.gen.go` in your service directory. This file contains package-level functions for the APIs defined as methods on the service struct. In the example above, you would see: ```go -- email/encore.gen.go -- // Code generated by encore. DO NOT EDIT. package email import "context" // These functions are automatically generated and maintained by Encore // to simplify calling them from other services, as they were implemented as methods. // They are automatically updated by Encore whenever your API endpoints change. func Send(ctx context.Context, p *SendParams) error { // The implementation is elided here, and generated at compile-time by Encore. return nil } ``` These functions are generated in order to allow other services to keep calling your APIs as package-level functions, in the same way as before: `email.Send(...)`. This means other services do not need to care about whether you're using Dependency Injection internally. You must always use these generated package-level functions for making API calls. > **Note** > > Encore will automatically generate these files and keep them up to date > whenever your code changes. There is no need to manually invoke anything > to regenerate this code. Encore adds all `encore.gen.go` files to your `.gitignore` since you typically don't want to commit them to your repository; doing so ends up creating a lot of unnecessary merge conflicts. However, in some cases when running third-party linters in a CI/CD environment it can be helpful to generate these wrappers to make the linter happy. You can do that by invoking `encore gen wrappers`. ## Graceful Shutdown When defining a service struct, Encore supports notifying your service when it's time to gracefully shut down. This works by having your service struct implement the method `func (s *Service) Shutdown(force context.Context)`. If that method exists, Encore will call it when it's time to begin gracefully shutting down. Initially the shutdown is in "graceful mode", which means that you have a few seconds to complete ongoing work. The provided `force` context is canceled when the graceful shutdown window is over, and it's time to forcefully shut down. How much time you have from when `Shutdown` is called to when forceful shutdown begins depends on the cloud provider and the underlying infrastructure. Typically it's in the range 5-30 seconds. > **Note** > > Encore automatically handles graceful shutdown of all Encore-managed > functionality, such as HTTP servers, database connection pools, > Pub/Sub message receivers, distributed tracing recorders, and so on. > > The graceful shutdown functionality is provided if you have additional, > non-Encore-related resources that need graceful shutdown. Note that graceful shutdown in Encore is *cooperative*: Encore will wait indefinitely for your `Shutdown` method to return. If your `Shutdown` method does not return promptly after the `force` context is closed, the underlying infrastructure at your cloud provider will typically force-kill your service, which can lead to lingering connections and other such issues. In summary, when your `Shutdown(force context.Context)` function is called: - Immediately begin gracefully shutting down - When the `force` context is canceled, you should forcefully shut down the resources that haven't yet completed their shutdown - Wait until the shutdown is complete before returning from the `Shutdown` function --- Source: https://encore.dev/docs/go/how-to/dependency-injection Section: Building (Encore.go) # Dependency Injection > Simplifying testing Dependency Injection is a fancy name for a simple concept: when you depend on some functionality, add that dependency as a field on your struct and refer to it that way instead of directly calling it. By doing so it becomes easier to test your services by swapping out certain dependencies for other implementations (often with the use of interfaces). Encore provides built-in support for dependency injection in services through the use of the `//encore:service` directive and a **service struct**. See the [service structs docs](https://encore.dev/docs/go/primitives/service-structs) more information on how to define service structs. As an example, consider an email service that has a SendGrid API client that is dependency injected. It might look like this: ```go package email //encore:service type Service struct { sendgridClient *sendgrid.Client } func initService() (*Service, error) { client, err := sendgrid.NewClient() if err != nil { return nil, err } return &Service{sendgridClient: client}, nil } ``` You can then define APIs as methods on this struct: ```go //encore:api private func (s *Service) Send(ctx context.Context, p *SendParams) error { // ... use s.sendgridClient to send emails ... } ``` ### Mocking dependencies If you wish to mock out the SendGrid client for testing purposes you can change the field to an interface: ```go type sendgridClient interface { SendEmail(...) // a hypothetical signature, for illustration purposes } //encore:service type Service struct { sendgridClient sendgridClient } ``` Then during your tests you can instantiate the service object by hand: ```go func TestFoo(t *testing.T) { svc := &Service{sendgridClient: &myMockClient{}} // ... } ``` --- Source: https://encore.dev/docs/go/how-to/http-requests Section: Building (Encore.go) # Receive regular HTTP requests > Dropping down in abstraction level Encore makes it easy to define APIs and expose them, but it works best when you are in charge of the API schema. Sometimes you need more control over the underlying HTTP request, such as to accept incoming webhooks from other services, or to use WebSockets to stream data to/from the client. For these use cases Encore lets you define **raw endpoints**. Raw endpoints operate at a lower abstraction level, giving you access to the underlying HTTP request. ## Defining raw endpoints To define a raw endpoint, change the `//encore:api` annotation and function signature like so: ```go package service import "net/http" // Webhook receives incoming webhooks from Some Service That Sends Webhooks. //encore:api public raw method=POST path=/webhook func Webhook(w http.ResponseWriter, req *http.Request) { // ... operate on the raw HTTP request ... } ``` If you're an experienced Go developer, this is just a regular Go HTTP handler. See the net/http documentation for more information on how Go HTTP handlers work. ## Reading path parameters Sometimes webhooks have information in the path that you may be interested in retrieving or validating. To do so, define the path with a path parameter, and then use [`encore.CurrentRequest`](https://pkg.go.dev/encore.dev#CurrentRequest) to access the path parameters. For example: ```go package service import ( "net/http" "encore.dev" ) //encore:api public raw method=POST path=/webhook/:id func Webhook(w http.ResponseWriter, req *http.Request) { id := encore.CurrentRequest().PathParams.Get("id") // ... Do something with id } ``` --- Source: https://encore.dev/docs/go/how-to/grpc-connect Section: Building (Encore.go) # Use Connect for incoming gRPC requests The [Connect protocol](https://connectrpc.com/) is an HTTP/2-based protocol for RPC communication. It's conceptually similar to gRPC, but with better support for using from browsers and JavaScript clients. This guide shows how to use Encore for setting up a Connect service for external clients to use: 1. First, we'll define a simple gRPC service using Protobuf and Connect. 2. Then, we'll implement the service in Go, using [connect-go](https://connectrpc.com/docs/go/getting-started). 3. Then, we'll mount the Connect service into Encore with a raw endpoint. 4. Finally, we'll call the Connect service from cURL using its JSON mapping. ## Define a Connect service We'll largely follow the connect-go [getting started guide](https://connectrpc.com/docs/go/getting-started) with some small tweaks. Start by installing the necessary tools: ```shell $ go install github.com/bufbuild/buf/cmd/buf@latest $ go install github.com/fullstorydev/grpcurl/cmd/grpcurl@latest $ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest $ go install connectrpc.com/connect/cmd/protoc-gen-connect-go@latest ``` Next, inside your Encore application ([create one if you haven't already](https://encore.dev/docs/quick-start)) create a new file at `greet/v1/greet.proto` with the following contents: ``` -- greet/v1/greet.proto -- syntax = "proto3"; package greet.v1; option go_package = "encore.app/gen/greet/v1;greetv1"; message GreetRequest { string name = 1; } message GreetResponse { string greeting = 1; } service GreetService { rpc Greet(GreetRequest) returns (GreetResponse) {} } ``` Next, add a `buf.gen.yaml` in the repository root, containing: ``` -- buf.gen.yaml -- version: v2 plugins: - local: protoc-gen-go out: gen opt: paths=source_relative - local: protoc-gen-connect-go out: gen opt: paths=source_relative ``` Now it's time to generate the connect-go service code. Run: ```shell $ buf lint $ buf generate ``` If all went well, you should see a new `gen` directory in the repository root containing some generated Go code: ``` gen └── greet └── v1 ├── greet.pb.go └── greetv1connect └── greet.connect.go ``` ## Implement the service Now that we have the service definition, we can implement the Connect service in Go. Add the file `greet/greet.go` with the following contents: ``` -- greet/greet.go -- package greet import ( "context" "fmt" "log" "connectrpc.com/connect" greetv1 "encore.app/gen/greet/v1" // generated by protoc-gen-go ) type GreetServer struct{} func (s *GreetServer) Greet( ctx context.Context, req *connect.Request[greetv1.GreetRequest], ) (*connect.Response[greetv1.GreetResponse], error) { log.Println("Request headers: ", req.Header()) res := connect.NewResponse(&greetv1.GreetResponse{ Greeting: fmt.Sprintf("Hello, %s!", req.Msg.Name), }) res.Header().Set("Greet-Version", "v1") return res, nil } ``` > **Note** > > The sample code is straight from the [getting started guide](https://connectrpc.com/docs/go/getting-started); > there are no Encore specific changes required here. ## Mount the service in Encore Now we'll create an Encore [service struct](https://encore.dev/docs/go/primitives/service-structs) that initializes the Connect service, and a [raw endpoint](https://encore.dev/docs/go/primitives/raw-endpoints) that forwards incoming requests to the Connect service. Add the file `greet/service.go` with the following contents: ``` -- greet/service.go -- package greet import ( "net/http" "encore.app/gen/greet/v1/greetv1connect" "golang.org/x/net/http2" "golang.org/x/net/http2/h2c" ) //encore:service type Service struct { routes http.Handler } //encore:api public raw path=/greet.v1.GreetService/*endpoint func (s *Service) GreetService(w http.ResponseWriter, req *http.Request) { s.routes.ServeHTTP(w, req) } func initService() (*Service, error) { greeter := &GreetServer{} mux := http.NewServeMux() path, handler := greetv1connect.NewGreetServiceHandler(greeter) mux.Handle(path, handler) return &Service{routes: mux}, nil } ``` That's it! We're ready to run the service and check that everything works. ## Run the service Run the service with `encore run`: ```shell $ encore run ``` Once it starts up, open a separate terminal and use `grpcurl` to call the service: ```shell # Install grpcurl if you haven't already $ go install github.com/fullstorydev/grpcurl/cmd/grpcurl@latest # Call the service with grpcurl grpcurl \ -protoset <(buf build -o -) -plaintext \ -d '{"name": "Jane"}' \ localhost:4000 greet.v1.GreetService/Greet {"greeting": "Hello, Jane!"} # Or call the service with curl $ curl -H "Content-Type: application/json" -d '{"name": "Jane"}' http://localhost:4000/greet.v1.GreetService/Greet {"greeting":"Hello, Jane!"} # Expected response ``` If you see `{"greeting":"Hello, Jane!"}`, everything is working! What's more, Encore automatically traces the incoming requests, and adds request logging and captures request metrics. --- Source: https://encore.dev/docs/go/primitives/code-snippets Section: Building (Encore.go) # Code snippets > Shortcuts for building with Encore When you're familiar with how Encore works, you can simplify your development workflow by copy-pasting these examples. If you're looking for details on how Encore works, please refer to the relevant docs section. ## APIs ### Defining APIs ```go package hello // service name //encore:api public func Ping(ctx context.Context, params *PingParams) (*PingResponse, error) { msg := fmt.Sprintf("Hello, %s!", params.Name) return &PingResponse{Message: msg}, nil } ``` ### Defining Request and Response schemas ```go // PingParams is the request data for the Ping endpoint. type PingParams struct { Name string } // PingResponse is the response data for the Ping endpoint. type PingResponse struct { Message string } ``` ### Calling APIs ```go import "encore.app/hello" // import service //encore:api public func MyOtherAPI(ctx context.Context) error { resp, err := hello.Ping(ctx, &hello.PingParams{Name: "World"}) if err == nil { log.Println(resp.Message) // "Hello, World!" } return err } ``` **Hint:** Import the service package and call the API endpoint using a regular function call. ### Receive Webhooks ```go import "net/http" // Webhook receives incoming webhooks from Some Service That Sends Webhooks. //encore:api public raw func Webhook(w http.ResponseWriter, req *http.Request) { // ... operate on the raw HTTP request ... } ``` **Hint:** Like any other API endpoint, this will be exposed at:
`https://-.encr.app/service.Webhook` ## Databases ### Creating a SQL database To create a database, import `encore.dev/storage/sqldb` and call `sqldb.NewDatabase`, assigning the result to a package-level variable. `sqldb.DatabaseConfig` specifies the directory containing the database migration files, which is how you define the database schema. ``` -- todo/db.go -- package todo // Create the todo database and assign it to the "tododb" variable var tododb = sqldb.NewDatabase("todo", sqldb.DatabaseConfig{ Migrations: "./migrations", }) // Then, query the database using db.QueryRow, db.Exec, etc. -- todo/migrations/1_create_table.up.sql -- CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL DEFAULT false -- etc... ); ``` ### Inserting data into a database One way of inserting data is with a helper function that calls `Exec` on the database: ```go import "encore.dev/storage/sqldb" // insert inserts a todo item into the database. func insert(ctx context.Context, id, title string, done bool) error { _, err := tododb.Exec(ctx, ` INSERT INTO todo_item (id, title, done) VALUES ($1, $2, $3) `, id, title, done) return err } ``` ### Querying a database To read a single todo item in the example schema above, we can use `sqldb.QueryRow`: ```go import "encore.dev/storage/sqldb" var item struct { ID int64 Title string Done bool } err := tododb.QueryRow(ctx, ` SELECT id, title, done FROM todo_item LIMIT 1 `).Scan(&item.ID, &item.Title, &item.Done) ``` **Hint:** If `sqldb.QueryRow` does not find a matching row, it reports an error that can be checked against by importing the standard library `errors` package and calling `errors.Is(err, sqldb.ErrNoRows)`. ## Defining a Cron Job ```go import "encore.dev/cron" var _ = cron.NewJob("welcome-email", cron.JobConfig{ Title: "Send welcome emails", Every: 2 * cron.Hour, Endpoint: SendWelcomeEmail, }) //encore:api private func SendWelcomeEmail(ctx context.Context) error { // ... return nil } ``` **Hint:** Cron Jobs do not run in your local development environment. ## PubSub ### Creating a PubSub topic ```go import "encore.dev/pubsub" type SignupEvent struct { UserID int } var Signups = pubsub.NewTopic[*SignupEvent]("signups", pubsub.TopicConfig { DeliveryGuarantee: pubsub.AtLeastOnce, }) ``` **Hint:** Topics are declared as package level variables and cannot be created inside functions. Regardless of where you create a topic, it can be published and subscribed to from any service. ### Publishing an Event (Pub) ```go if _, err := Signups.Publish(ctx, &SignupEvent{UserID: id}); err != nil { return err } if err := tx.Commit(); err != nil { return err } ``` **Hint:** If you want to publish to the topic from another service, import the topic package variable (`Signups` in this example) and call publish on it from there. ### Subscribing to Events (Sub) Create a Subscription as a package level variable by calling `pubsub.NewSubscription`. ```go var _ = pubsub.NewSubscription( user.Signups, "send-welcome-email", pubsub.SubscriptionConfig[*SignupEvent] { Handler: SendWelcomeEmail, }, ) func SendWelcomeEmail(ctx context.Context, event *SignupEvent) error { ... send email ... return nil } ``` ## Defining a Cache cluster ```go import "encore.dev/storage/cache" var MyCacheCluster = cache.NewCluster("my-cache-cluster", cache.ClusterConfig{ // EvictionPolicy tells Redis how to evict keys when the cache reaches // its memory limit. For typical cache use cases, cache.AllKeysLRU is a good default. EvictionPolicy: cache.AllKeysLRU, }) ``` ## Secrets ### Defining Secrets ```go var secrets struct { GitHubAPIToken string // personal access token for deployments SomeOtherSecret string // some other secret } ``` **Hint:** The variable must be an unexported struct named `secrets`, and all the fields must be of type `string`. ### Setting secret values ```shell $ encore secret set --type ``` **Hint:** `` defines which environment types the secret value applies to. Use a comma-separated list of `production`, `development`, `preview`, and `local`. For each Secret, there can only be one secret value for each environment type. ### Using secrets ```go func callGitHub(ctx context.Context) { req, _ := http.NewRequestWithContext(ctx, "GET", "https:///api.github.com/user", nil) req.Header.Add("Authorization", "token " + secrets.GitHubAPIToken) resp, err := http.DefaultClient.Do(req) // ... handle err and resp } ``` **Hint:** Secret keys are globally unique for your whole application; if multiple services use the same secret name they both receive the same secret value at runtime. --- Source: https://encore.dev/docs/go/develop/auth Section: Building (Encore.go) # Authenticating users > Knowing what's what and who's who Almost every application needs to know who's calling it, whether the user represents a person in a consumer-facing app or an organization in a B2B app. Encore supports both use cases in a simple yet powerful way. As described in the docs for [defining APIs](https://encore.dev/docs/go/primitives/defining-apis), Encore offers three access levels for APIs: * `//encore:api public` – defines a public API that anybody on the internet can call. * `//encore:api private` – defines a private API that is never accessible to the outside world. It can only be called from other services in your app and via cron jobs. * `//encore:api auth` – defines a public API that anybody can call, but that requires valid authentication. When an API is defined with access level `auth`, outside calls to that API must specify an authorization header, in the form `Authorization: Bearer `. The token is passed to a designated auth handler function and the API call is allowed to go through only if the auth handler determines the token is valid. For more advanced use cases you can also customize the authentication information you want. See the section on [accepting structured auth information](#accepting-structured-auth-information) below. > **Note** > > You can optionally send in auth data to `public` and `private` APIs, in which case the auth handler will be used. When used for `private` APIs, they are still not accessible from the outside world. ## The auth handler Encore applications can designate a special function to handle authentication, by defining a function and annotating it with `//encore:authhandler`. This annotation tells Encore to run the function whenever an incoming API call contains authentication data. The auth handler is responsible for validating the incoming authentication data and returning an `auth.UID` (a string type representing a **user id**). The `auth.UID` can be whatever you wish, but in practice it usually maps directly to the primary key stored in a user table (either defined in the Encore service or in an external service like [Firebase](https://encore.dev/docs/go/how-to/firebase-auth) or [Auth0](https://encore.dev/docs/go/how-to/auth0-auth)). ### With custom user data Oftentimes it's convenient for the rest of your application to easily be able to look up information about the authenticated user making the request. If that's the case, define the auth handler like so: ```go import "encore.dev/beta/auth" // Data can be named whatever you prefer (but must be exported). type Data struct { Username string // ... } // AuthHandler can be named whatever you prefer (but must be exported). //encore:authhandler func AuthHandler(ctx context.Context, token string) (auth.UID, *Data, error) { // Validate the token and look up the user id and user data, // for example by calling Firebase Auth. } ``` **Related example:** [encoredev/examples/clerk](https://github.com/encoredev/examples/tree/main/clerk) Example application showing an auth handler implementation with Clerk. Create it locally with `encore app create --example=clerk`. **Related docs:** - [/docs/go/how-to/auth0-auth](https://encore.dev/docs/go/how-to/auth0-auth) - [/docs/go/how-to/clerk-auth](https://encore.dev/docs/go/how-to/clerk-auth) - [/docs/go/how-to/firebase-auth](https://encore.dev/docs/go/how-to/firebase-auth) ### Without custom user data When you don't require custom user data and it's sufficient to use `auth.UID`, simply skip it in the return type: ```go import "encore.dev/beta/auth" // AuthHandler can be named whatever you prefer (but must be exported). //encore:authhandler func AuthHandler(ctx context.Context, token string) (auth.UID, error) { // Validate the token and look up the user id, // for example by calling Firebase Auth. } ``` ## Accepting structured auth information In the examples above the function accepts a `Bearer` token as a string argument. In that case Encore parses the `Authorization` HTTP header and passes the token to the auth handler. In cases where you have different or more complex authorization requirements, you can instead specify a data structure that specifies one or more fields to be parsed from the HTTP request. For example: ```go type MyAuthParams struct { // SessionCookie is set to the value of the "session" cookie. // If the cookie is not set it's nil. SessionCookie *http.Cookie `cookie:"session"` // ClientID is the unique id of the client, sourced from the URL query string. ClientID string `query:"client_id"` // Authorization is the raw value of the "Authorization" header // without any parsing. Authorization string `header:"Authorization"` } //encore:authhandler func AuthHandler(ctx context.Context, p *MyAuthParams) (auth.UID, error) { // ... } ``` This example tells Encore that the application accepts authentication information via the `session` cookie, the `client_id` query string parameter, and the `Authorization` header. These fields are automatically filled in when the auth handler is called (if present in the request). You can of course combine auth params like this with custom user data (see the section above). > **Note** > > Cookies are generally only used by browsers and are automatically added to requests made by browsers. > As a result Encore does not include cookie fields in generated clients' authentication payloads > or in the [Local Development Dashboard](https://encore.dev/docs/dev-dash). ## Handling auth errors When a token doesn't match your auth rules (for example if it's expired, the token has been revoked, or the token is invalid), you should return a non-nil error from the auth handler. Encore passes the error message on to the user when you use [Encore's built-in error package](https://encore.dev/docs/go/primitives/api-errors), so we recommend using that with the error code `Unauthenticated` to communicate what happened. For example: ```go import "encore.dev/beta/errs" //encore:authhandler func AuthHandler(ctx context.Context, token string) (auth.UID, error) { return "", &errs.Error{ Code: errs.Unauthenticated, Message: "invalid token", } } ``` > **Important** > > Note that for security reasons you may not want to reveal too much information about why a request did not pass your auth checks. There are many subtle security considerations when dealing with authentication and we don't have time to go into all of them here. > > Whenever possible we recommend using a third-party auth provider instead of rolling your own authentication. ## Using auth data Once the user has been identified by the auth handler, the API handler is called as usual. If it wishes to inspect the authenticated user, it can use the `encore.dev/beta/auth` package: - `auth.Data()` returns the custom user data returned by the auth handler (if any) - `auth.UserID()` returns `(auth.UID, bool)` to get the authenticated user id (if any) For an incoming request from the outside to an API that uses the `auth` access level, these are guaranteed to be set since the API won't be called if the auth handler doesn't succeed. Encore automatically propagates the auth data when you make API calls to other Encore API endpoints. > **Note** > > If an endpoint calls another endpoint during its processing, and the original > does not have an authenticated user, the request will fail. This behavior > preserves the guarantees that `auth` endpoints always have an authenticated user. ## Optional authentication While Encore always calls the auth handler for API endpoints marked as `auth`, you can also call `public` API endpoints with authentication data. This can be useful for APIs that support both a "logged in" and "logged out" experience. For example, a site like Reddit might have a `post.List` endpoint that returns the list of posts, but if you're logged in it also includes whether or not you have upvoted or downvoted each post. To support such use cases, Encore runs the auth handler for `public` API endpoints if (and only if) the request includes any authentication information (such as the `Authorization` header). In that case, the request processing behavior varies depending on the value of the `error` returned from the auth handler: * If the error is nil, the request is considered to be an authenticated request and `auth.UID()` and `auth.Data()` will include the information the auth handler returned. * If the error is non-nil and the error code is `errs.Unauthenticated` (like shown above), the request continues as an unauthenticated request, behaving exactly as if there was no authentication data provided at all. * If the error is non-nil and the error code is anything else, the request is aborted and Encore returns that error to the caller. To be able to determine if the request has an authenticated user, check the second return value from `auth.UserID()`. ## Overriding auth information Encore supports overriding the auth information for an outgoing request using the [`auth.WithContext`](https://pkg.go.dev/encore.dev/beta/auth#WithContext) function. This function returns a new context with the auth information set to the specified values. Note that this only affects the auth information passed along with the request, and not the current request being processed (if any). This function is often useful when testing APIs that use authentication. For example: ```go ctx := auth.WithContext(context.Background(), auth.UID("my-user-id"), &MyAuthData{Email: "hello@example.com"}) // ... Make an API call using `ctx` to override the auth information for that API call. ``` --- Source: https://encore.dev/docs/go/develop/middleware Section: Building (Encore.go) # Middleware > Handling cross-cutting, generic functionality Middleware is a way to write reusable code that runs before or after (or both) the handling of API requests, often across several (or all) API endpoints. It's commonly used to implement cross-cutting concerns like [request logging](https://encore.dev/docs/go/observability/logging), [authentication](https://encore.dev/docs/go/develop/auth), [tracing](https://encore.dev/docs/tracing), and so on. One of the benefits of Encore is that all of these use cases are already handled out-of-the-box, so there's no need to use middleware for those things. Nonetheless, there are several use cases where it can be useful to write reusable functionality that applies to multiple API endpoints, and middleware is a good solution in those cases. Encore provides built-in support for middleware by defining a function with the `//encore:middleware` directive. The middleware directive takes a `target` parameter that specifies which API endpoints it applies to. ## Middleware functions A typical middleware implementation looks like this: ```go import ( "encore.dev/beta/errs" "encore.dev/middleware" ) //encore:middleware global target=all func ValidationMiddleware(req middleware.Request, next middleware.Next) middleware.Response { // If the payload has a Validate method, use it to validate the request. payload := req.Data().Payload if validator, ok := payload.(interface { Validate() error }); ok { if err := validator.Validate(); err != nil { // If the validation fails, return an InvalidArgument error. err = errs.WrapCode(err, errs.InvalidArgument, "validation failed") return middleware.Response{Err: err} } } return next(req) } ``` Middleware forms a chain, allowing each middleware to introspect and process the incoming request before handing it off to the next middleware by calling the `next` function that's passed in as an argument. For the last middleware in the chain, calling `next` results in the actual API handler being called. The `req` parameter provides information about the incoming request (see [package docs](https://pkg.go.dev/encore.dev/middleware#Request)). The `next` function returns a [`middleware.Response`](https://pkg.go.dev/encore.dev/middleware#Response) object which contains the response from the API, describing whether there was an error, and on success the actual response payload. This enables middleware to also introspect and even modify the outgoing response, like this: ```go //encore:middleware target=tag:cache func CachingMiddleware(req middleware.Request, next middleware.Next) middleware.Response { data := req.Data() // Check if we have the response cached. Use the request path as the cache key. cacheKey := data.Path if cached, err := loadFromCache(cacheKey, data.API.ResponseType); err == nil && cached != nil { return middleware.Response{Payload: cached} } // Otherwise forward the request to the handler return next(req) } ``` This uses `target=tag:cache` to have the middleware only apply to APIs that have that tag. More on this below in [Targeting APIs](#targeting-apis). > **Important** > > Middleware functions can also be defined as methods on a Dependency Injection > struct declared with `//encore:service`. For example: > > ```go > //encore:service > type Service struct{} > > //encore:middleware target=all > func (s *Service) MyMiddleware(req middleware.Request, next middleware.Next) middleware.Response { > // ... > } > ``` > > See the [Dependency Injection](https://encore.dev/docs/go/how-to/dependency-injection) docs for more information. ## Middleware ordering Middleware can either be defined inside a service, in which case it only runs for APIs within that service, or it can be defined as a `global` middleware, in which case it applies to all services. For global middleware the `target` directive still applies and enables you to easily match a subset of APIs. > **Important** > > Global middleware always run before all service-specific middleware, > and then run in the order they are defined in the source code based on > file name lexicographic ordering. To avoid surprises it's best to define all middleware in a file called `middleware.go` in each service, and to create a single top-level package to contain all global middleware. ## Targeting APIs The `target` directive can either be provided as `target=all` (meaning it applies to all APIs) or a list of tags, in the form `target=tag:foo,tag:bar`. Note that these tags are evaluated with `OR`, meaning the middleware applies to an API if the API has at least one of those tags. APIs can be defined with tags by adding `tag:foo` at the end of the `//encore:api` directive: ```go //encore:api public method=GET path=/user/:id tag:cache func GetUser(ctx context.Context, id string) (*User, error) { // ... } ``` --- Source: https://encore.dev/docs/go/how-to/auth0-auth Section: Building (Encore.go) # Use Auth0 with your app In this guide you will learn how to set up an Encore [auth handler](https://encore.dev/docs/go/develop/auth#the-auth-handler) that makes use of [Auth0](https://auth0.com/) in order to add a seamless signup and login experience to your web app. For all the code and instructions of how to clone and run this example locally, see the [Auth0 Example](https://github.com/encoredev/examples/tree/main/auth0) in our examples repo. ## Communicate with Auth0 In your Encore app, install two modules: ```shell $ go get github.com/coreos/go-oidc/v3/oidc golang.org/x/oauth2 ``` Create a folder and naming it `auth`, this is where our authentication related backend code will live. Next, let's set up the Auth0 `Authenticator` that will be used by our auth handler. The `Authenticator` has a method to configure and return [OAuth2](https://pkg.go.dev/golang.org/x/oauth2?utm_source=godoc) and [oidc](https://pkg.go.dev/github.com/coreos/go-oidc?utm_source=godoc) clients, and another one to verify an ID Token. Create `auth/authenticator.go` and paste the following: ```go -- auth/authenticator.go -- package auth import ( "context" "crypto/rand" "encoding/base64" "encore.dev/config" "errors" "github.com/coreos/go-oidc/v3/oidc" "golang.org/x/oauth2" ) type Auth0Config struct { ClientID config.String Domain config.String CallbackURL config.String LogoutURL config.String } var cfg = config.Load[*Auth0Config]() var secrets struct { Auth0ClientSecret string } // Authenticator is used to authenticate our users. type Authenticator struct { *oidc.Provider oauth2.Config } // New instantiates the *Authenticator. func New() (*Authenticator, error) { provider, err := oidc.NewProvider( context.Background(), "https://"+cfg.Domain()+"/", ) if err != nil { return nil, err } conf := oauth2.Config{ ClientID: cfg.ClientID(), ClientSecret: secrets.Auth0ClientSecret, RedirectURL: cfg.CallbackURL(), Endpoint: provider.Endpoint(), Scopes: []string{oidc.ScopeOpenID, "profile", "email"}, } return &Authenticator{ Provider: provider, Config: conf, }, nil } // VerifyIDToken verifies that an *oauth2.Token is a valid *oidc.IDToken. func (a *Authenticator) VerifyIDToken(ctx context.Context, token *oauth2.Token) (*oidc.IDToken, error) { rawIDToken, ok := token.Extra("id_token").(string) if !ok { return nil, errors.New("no id_token field in oauth2 token") } oidcConfig := &oidc.Config{ ClientID: a.ClientID, } return a.Verifier(oidcConfig).Verify(ctx, rawIDToken) } func generateRandomState() (string, error) { b := make([]byte, 32) _, err := rand.Read(b) if err != nil { return "", err } state := base64.StdEncoding.EncodeToString(b) return state, nil } ``` ## Set up the auth handler It's time to define your [auth handler](https://encore.dev/docs/go/develop/auth) and the endpoints needed for the login and logout flow. Create the `auth/auth.go` file and paste the following: ```go -- auth/auth.go -- package auth import ( "context" "net/url" "encore.dev/beta/auth" "encore.dev/beta/errs" "github.com/coreos/go-oidc/v3/oidc" ) // Service struct definition. // Learn more: encore.dev/docs/primitives/services-and-apis/service-structs // //encore:service type Service struct { auth *Authenticator } // initService is automatically called by Encore when the service starts up. func initService() (*Service, error) { authenticator, err := New() if err != nil { return nil, err } return &Service{auth: authenticator}, nil } type LoginResponse struct { State string `json:"state"` AuthCodeURL string `json:"auth_code_url"` } //encore:api public method=POST path=/auth/login func (s *Service) Login(ctx context.Context) (*LoginResponse, error) { state, err := generateRandomState() if err != nil { return nil, &errs.Error{ Code: errs.Internal, Message: err.Error(), } } return &LoginResponse{ State: state, // add the audience to the auth code url AuthCodeURL: s.auth.AuthCodeURL(state), }, nil } type CallbackRequest struct { Code string `json:"code"` } type CallbackResponse struct { Token string `json:"token"` } //encore:api public method=POST path=/auth/callback func (s *Service) Callback( ctx context.Context, req *CallbackRequest, ) (*CallbackResponse, error) { // Exchange an authorization code for a token. token, err := s.auth.Exchange(ctx, req.Code) if err != nil { return nil, &errs.Error{ Code: errs.PermissionDenied, Message: "Failed to convert an authorization code into a token.", } } idToken, err := s.auth.VerifyIDToken(ctx, token) if err != nil { return nil, &errs.Error{ Code: errs.Internal, Message: "Failed to verify ID Token.", } } var profile map[string]interface{} if err := idToken.Claims(&profile); err != nil { return nil, &errs.Error{ Code: errs.Internal, Message: err.Error(), } } return &CallbackResponse{ Token: token.Extra("id_token").(string), }, nil } type LogoutResponse struct { RedirectURL string `json:"redirect_url"` } //encore:api public method=GET path=/auth/logout func (s *Service) Logout(ctx context.Context) (*LogoutResponse, error) { logoutUrl, err := url.Parse("https://" + cfg.Domain() + "/v2/logout") if err != nil { return nil, &errs.Error{ Code: errs.Internal, Message: err.Error(), } } returnTo, err := url.Parse(cfg.LogoutURL()) if err != nil { return nil, &errs.Error{ Code: errs.Internal, Message: err.Error(), } } parameters := url.Values{} parameters.Add("returnTo", returnTo.String()) parameters.Add("client_id", cfg.ClientID()) logoutUrl.RawQuery = parameters.Encode() return &LogoutResponse{ RedirectURL: logoutUrl.String(), }, nil } type ProfileData struct { Email string `json:"email"` Picture string `json:"picture"` } // The `encore:authhandler` annotation tells Encore to run this function for all // incoming API call that requires authentication. // Learn more: encore.dev/docs/develop/auth#the-auth-handler // //encore:authhandler func (s *Service) AuthHandler( ctx context.Context, token string, ) (auth.UID, *ProfileData, error) { oidcConfig := &oidc.Config{ ClientID: s.auth.ClientID, } t, err := s.auth.Verifier(oidcConfig).Verify(ctx, token) if err != nil { return "", nil, &errs.Error{ Code: errs.Unauthenticated, Message: "invalid token", } } var profile map[string]interface{} if err := t.Claims(&profile); err != nil { return "", nil, &errs.Error{ Code: errs.Internal, Message: err.Error(), } } // Extract profile data returned from the identity provider. // auth0.com/docs/manage-users/user-accounts/user-profiles/user-profile-structure profileData := &ProfileData{ Email: profile["email"].(string), Picture: profile["picture"].(string), } return auth.UID(profile["sub"].(string)), profileData, nil } // Endpoints annotated with `auth` are public and requires authentication // Learn more: encore.dev/docs/primitives/apis#access-controls // //encore:api auth method=GET path=/profile func GetProfile(ctx context.Context) (*ProfileData, error) { return auth.Data().(*ProfileData), nil } ``` ## Auth0 settings The `Authenticator` class requires some values that are specific your Auth0 application, namely the `ClientID`, `ClientSecret`, `Domain`, `CallbackURL` and `LogoutURL`. Create an Auth0 account if you haven't already. Then, in the Auth0 dashboard, create a new *Single Page Web Applications*. ![Create Auth0 application](https://encore.dev/assets/docs/auth0-create-app.png) Next, go to the *Application Settings* section. There you will find the `Domain`, `Client ID`, and `Client Secret` that you need to communicate with Auth0. Copy these values, we will need them shortly. ![Auth0 basic information](https://encore.dev/assets/docs/auth0-basic-info.png) A callback URL is where Auth0 redirects the user after they have been authenticated. Add `http://localhost:3000/callback` to the *Allowed Callback URLs*. You will need to add more URLs to this list when you have a production or staging environments. The same goes for the logout URL (were the user will get redirected after logout). Add `http://localhost:3000/` to the *Allowed Logout URLs*. ![Auth0 application URIs](https://encore.dev/assets/docs/auth0-app-uris.png) ## Config and secrets Create a [configuration file](https://encore.dev/docs/go/develop/config) in the `auth` service and name it `auth-config.cue`. Add the following: ```cue -- auth/auth-config.cue -- ClientID: "" Domain: "" // An application running locally if #Meta.Environment.Type == "development" && #Meta.Environment.Cloud == "local" { CallbackURL: "http://localhost:3000/callback" LogoutURL: "http://localhost:3000/" } ``` Replace the values for the `ClientID` and `Domain` that you got from the Auth0 dashboard. The `ClientSecret` is especially sensitive and should not be hardcoded in your code/config. Instead, you should store that as an [Encore secret](https://encore.dev/docs/go/primitives/secrets). From your terminal (inside your Encore app directory), run: ```shell $ encore secret set --prod Auth0ClientSecret ``` Now you should do the same for the development secret. The most secure way is to set up a different Auth0 application and use that for development. Depending on your security requirements you could also use the same secret for development and production. Once you have a client secret for development, set it similarly to before: ```shell $ encore secret set --dev Auth0ClientSecret ``` That's it! Encore will run your auth handler and validate the token against Auth0. ## Frontend Now that the backend is set up, we can create a frontend application that uses the login flow. Here's an example using [React](https://react.dev/) together with [React Router](https://reactrouter.com/). This example also makes use of a Encores ability to [generate request clients](https://encore.dev/docs/go/cli/client-generation) to make the communication with our backend simple and typesafe. ```tsx -- App.tsx -- import { PropsWithChildren } from "react"; import { createBrowserRouter, Link, Outlet, redirect, RouterProvider, useRouteError, } from "react-router-dom"; import { Auth0Provider } from "./lib/auth"; import AdminDashboard from "./components/AdminDashboard.tsx"; import IndexPage from "./components/IndexPage.tsx"; import "./App.css"; import LoginStatus from "./components/LoginStatus.tsx"; // Application routes const router = createBrowserRouter([ { id: "root", path: "/", Component: Layout, errorElement: ( ), children: [ { Component: Outlet, children: [ { index: true, Component: IndexPage, }, { // Login route path: "login", loader: async ({ request }) => { const url = new URL(request.url); const searchParams = new URLSearchParams(url.search); const returnToURL = searchParams.get("returnTo") ?? "/"; if (Auth0Provider.isAuthenticated()) return redirect(returnToURL); try { const returnURL = await Auth0Provider.login(returnToURL); return redirect(returnURL); } catch (error) { throw new Error("Login failed"); } }, }, { // Callback route, redirected to from Auth0 after login path: "callback", loader: async ({ request }) => { const url = new URL(request.url); const searchParams = new URLSearchParams(url.search); const state = searchParams.get("state"); const code = searchParams.get("code"); if (!state || !code) throw new Error("Login failed"); try { const redirectURL = await Auth0Provider.validate(state, code); return redirect(redirectURL); } catch (error) { throw new Error("Login failed"); } }, }, { // Logout route path: "logout", loader: async () => { try { const redirectURL = await Auth0Provider.logout(); return redirect(redirectURL); } catch (error) { throw new Error("Logout failed"); } }, }, { element: , // Redirect to /login if not authenticated loader: async ({ request }) => { if (!Auth0Provider.isAuthenticated()) { const params = new URLSearchParams(); params.set("returnTo", new URL(request.url).pathname); return redirect("/login?" + params.toString()); } return null; }, // Protected routes children: [ { path: "admin-dashboard", Component: AdminDashboard, }, ], }, ], }, ], }, ]); export default function App() { return Loading...

} />; } function Layout({ children }: PropsWithChildren) { return (
{children ?? }
); } function ErrorBoundary() { const error = useRouteError() as Error; return (

Something went wrong

{error.message || JSON.stringify(error)}

); } -- lib/auth.ts -- import Cookies from "js-cookie"; import getRequestClient from "./getRequestClient.ts"; type RedirectURL = string; /** * Handles the backend communication for the authentication flow. */ export const Auth0Provider = { client: getRequestClient(), isAuthenticated: () => !!Cookies.get("auth-token"), async login(returnTo: RedirectURL): Promise { const response = await this.client.auth.Login(); Cookies.set("state", response.state); sessionStorage.setItem(response.state, returnTo); return response.auth_code_url; }, async logout(): Promise { const response = await this.client.auth.Logout(); Cookies.remove("auth-token"); Cookies.remove("state"); return response.redirect_url; }, async validate(state: string, authCode: string): Promise { if (state != Cookies.get("state")) throw new Error("Invalid state"); const response = await this.client.auth.Callback({ code: authCode }); Cookies.set("auth-token", response.token); const returnURL = sessionStorage.getItem(state) ?? "/"; sessionStorage.removeItem(state); return returnURL; }, }; -- components/LoginStatus.tsx -- import getRequestClient from "../lib/getRequestClient.ts"; import { useFetcher } from "react-router-dom"; import { useEffect, useState } from "react"; import { auth } from "../lib/client.ts"; import { Auth0Provider } from "../lib/auth.ts"; /** * Component displaying login/logout button and basic user information if logged in. */ function LoginStatus() { const client = getRequestClient(); const fetcher = useFetcher(); const [profile, setProfile] = useState(); const [loading, setLoading] = useState(true); // Fetch profile data if user is authenticated useEffect(() => { const getProfile = async () => { setProfile(await client.auth.GetProfile()); setLoading(false); }; if (Auth0Provider.isAuthenticated()) getProfile(); else setLoading(false); }, []); if (loading) return null; if (profile) { return (
); } const params = new URLSearchParams(); params.set("returnTo", window.location.pathname); return (
); } export default LoginStatus; -- lib/getRequestClient.ts -- import Client, { Environment, Local } from "./client.ts"; import Cookies from "js-cookie"; /** * Returns the generated Encore request client for either the local or staging environment. * If we are running the frontend locally (development) we assume that our Encore * backend is also running locally. */ const getRequestClient = () => { const token = Cookies.get("auth-token"); const env = import.meta.env.DEV ? Local : Environment("staging"); return new Client(env, { auth: token, }); }; export default getRequestClient; ``` ## Auth0 Social Identity Providers Auth0 supports multiple [social identity providers](https://auth0.com/docs/authenticate/identity-providers/social-identity-providers) (like Google and GitHub) for web applications out of the box. --- Source: https://encore.dev/docs/go/how-to/clerk-auth Section: Building (Encore.go) # Use Clerk with your app In this guide you will learn how to set up an Encore [auth handler](https://encore.dev/docs/go/develop/auth#the-auth-handler) that makes use of [Clerk](https://clerk.com/) in order to add an integrated signup and login experience to your web app. For all the code and instructions of how to clone and run this example locally, see the [Clerk Example](https://github.com/encoredev/examples/tree/main/clerk) in our examples repo. **Deploy to Encore** Deploy this app to a free development environment. - [Deploy](https://app.encore.dev/create-app/clone/clerk) ## Set up the auth handler In your Encore app, install the following module: ```shell $ go get github.com/clerkinc/clerk-sdk-go/clerk ``` Create a folder and naming it `auth`, this is where our authentication related backend code will live. It's time to define your [auth handler](https://encore.dev/docs/go/develop/auth). Create `auth/auth.go` and paste the following: ```go -- auth/auth.go -- package auth import ( "context" "encore.dev/beta/auth" "encore.dev/beta/errs" "github.com/clerkinc/clerk-sdk-go/clerk" ) var secrets struct { ClientSecretKey string } // Service struct definition. // Learn more: encore.dev/docs/primitives/services-and-apis/service-structs // //encore:service type Service struct { client clerk.Client } // initService is automatically called by Encore when the service starts up. func initService() (*Service, error) { client, err := clerk.NewClient(secrets.ClientSecretKey) if err != nil { return nil, err } return &Service{client: client}, nil } type UserData struct { ID string `json:"id"` Username *string `json:"username"` FirstName *string `json:"first_name"` LastName *string `json:"last_name"` ProfileImageURL string `json:"profile_image_url"` PrimaryEmailAddressID *string `json:"primary_email_address_id"` EmailAddresses []clerk.EmailAddress `json:"email_addresses"` } // The `encore:authhandler` annotation tells Encore to run this function for all // incoming API call that requires authentication. // Learn more: encore.dev/docs/develop/auth#the-auth-handler // //encore:authhandler func (s *Service) AuthHandler(ctx context.Context, token string) (auth.UID, *UserData, error) { // verify the session sessClaims, err := s.client.VerifyToken(token) if err != nil { return "", nil, &errs.Error{ Code: errs.Unauthenticated, Message: "invalid token", } } user, err := s.client.Users().Read(sessClaims.Claims.Subject) if err != nil { return "", nil, &errs.Error{ Code: errs.Internal, Message: err.Error(), } } userData := &UserData{ ID: user.ID, Username: user.Username, FirstName: user.FirstName, LastName: user.LastName, ProfileImageURL: user.ProfileImageURL, PrimaryEmailAddressID: user.PrimaryEmailAddressID, EmailAddresses: user.EmailAddresses, } return auth.UID(user.ID), userData, nil } ``` ## Clerk credentials Create a Clerk account if you haven't already. Then, in the Clerk dashboard, create a new applications. Next, go to the *API Keys* page for your app. Copy one of the "Secret keys" (the "Publishable Key" will be used by your frontend). The `Secret key` is sensitive and should not be hardcoded in your code/config. Instead, you should store that as an [Encore secret](https://encore.dev/docs/go/primitives/secrets). From your terminal (inside your Encore app directory), run: ```shell $ encore secret set --prod ClientSecretKey ``` Now you should do the same for the development secret. The most secure way is to create another secret key (Clerk allows you to have multiple). Once you have a client secret for development, set it similarly to before: ```shell $ encore secret set --dev ClientSecretKey ``` ## Frontend Clerk offers a [React SDK](https://clerk.com/docs/references/react/overview) for the frontend which makes it really simple to integrate a login/signup flow inside your web app as well as getting the token required to communicate with your Encore backend. You can use the `useAuth` hook from `@clerk/clerk-react` to get the token and send it to your backend. ```tsx import { useAuth } from '@clerk/clerk-react'; export default function ExternalDataPage() { const { getToken, isLoaded, isSignedIn } = useAuth(); if (!isLoaded) { // Handle loading state however you like return
Loading...
; } if (!isSignedIn) { // Handle signed out state however you like return
Sign in to view this page
; } const fetchDataFromExternalResource = async () => { const token = await getToken(); // Use token to send to Encore backend when fetching data return data; } return
...
; } ``` For a fully working backend + frontend example see the [Clerk Example](https://github.com/encoredev/examples/tree/main/clerk) in our examples repo. --- Source: https://encore.dev/docs/go/how-to/firebase-auth Section: Building (Encore.go) # Use Firebase Auth with your app Encore's [authentication support](https://encore.dev/docs/go/develop/auth) provides a simple yet powerful way of dealing with various authentication scenarios. Firebase Authentication is a common solution for quickly setting up a user store and simplifying social logins. Encore makes it really easy to integrate with Firebase Authentication on the backend. For all the code and instructions of how to clone and run this example locally, see the [Firebase Auth Example](https://github.com/encoredev/examples/tree/main/firebase-auth) in our examples repo. ## Set up auth handler First, install two modules: ```shell $ go get firebase.google.com/go/v4 go4.org/syncutil ``` Next it's time to define your [authentication handler](https://encore.dev/docs/go/develop/auth). It can live in whatever service you'd like, but it's usually easiest to create a designated `user` service. Create the `user/user.go` file and add the following skeleton code: ```go -- user/user.go -- package user import ( "context" "strings" "encore.dev/beta/auth" firebase "firebase.google.com/go/v4" fbauth "firebase.google.com/go/v4/auth" "go4.org/syncutil" "google.golang.org/api/option" ) // Data represents the user's data stored in Firebase Auth. type Data struct { // Email is the user's email. Email string // Name is the user's name. Name string // Picture is the user's picture URL. Picture string } // ValidateToken validates an auth token against Firebase Auth. //encore:authhandler func ValidateToken(ctx context.Context, token string) (auth.UID, *Data, error) { panic("Not Yet Implemented") } ``` ## Initialize Firebase SDK Next, let's set up the Firebase Auth client. We'll use  `syncutil.Once`  to do it lazily the first time we need it. Add to the bottom of our file: ```go -- user/user.go -- var ( fbAuth *fbauth.Client setupOnce syncutil.Once ) // setupFB ensures Firebase Auth is setup. func setupFB() error { return setupOnce.Do(func() error { opt := option.WithCredentialsJSON([]byte(secrets.FirebasePrivateKey)) app, err := firebase.NewApp(context.Background(), nil, opt) if err == nil { fbAuth, err = app.Auth(context.Background()) } return err }) } var secrets struct { // FirebasePrivateKey is the JSON credentials for calling Firebase. FirebasePrivateKey string } ``` ## Validate token against Firebase Now that we have the code to initialize Firebase Auth, we can use it from our `ValidateToken` auth handler. Update the function to look like the following: ```go func ValidateToken(ctx context.Context, token string) (auth.UID, *Data, error) { if err := setupFB(); err != nil { return "", nil, err } tok, err := fbAuth.VerifyIDToken(ctx, token) if err != nil { return "", nil, err } email, _ := tok.Claims["email"].(string) name, _ := tok.Claims["name"].(string) picture, _ := tok.Claims["picture"].(string) uid := auth.UID(tok.UID) usr := &Data{ Email: email, Name: name, Picture: picture, } return uid, usr, nil } ``` Great! We're done with the code. Now we just need to set up the secret. ## Set Firebase secret credentials If you haven't already, set up a Firebase project. Then, go to **Project settings** and navigate to **Service accounts**. Select `Go` as the language of choice and click `Generate new private key`. Download the generated key and take note where it is stored. Next, store the private key as your firebase secret. From your terminal (inside your Encore app directory), run: ```shell $ encore secret set --type prod FirebasePrivateKey < /path/to/firebase-private-key.json Successfully updated production secret FirebasePrivateKey ``` Now you should do the same for the development secret. The most secure way is to set up a different Firebase project and use that for development. Depending on your security requirements you could also use the same Firebase project, but we recommend generating a new private key for development in that case. Once you have a private key for development, set it similarly to before: ```shell $ encore secret set --type dev,local,pr FirebasePrivateKey < /path/to/firebase-private-key.json Successfully updated development secret FirebasePrivateKey ``` That's it! You can now call your Encore application and pass in Firebase tokens. Encore will run your auth handler and validate the token against Firebase Auth. ## Frontend Firebase offers a [npm package](https://www.npmjs.com/package/firebase) for your web frontend which makes it really simple to create a login/signup flow inside your web app as well as getting the token required to communicate with your Encore backend. For a fully working backend + frontend example see the [Firebase Auth Example](https://github.com/encoredev/examples/tree/main/firebase-auth) in our examples repo. --- Source: https://encore.dev/docs/go/how-to/logto-auth Section: Building (Encore.go) # Use Logto with your app [Logto](https://logto.io) is a modern Auth0 alternative that helps you build the sign-in experience and user identity within minutes. It's particularly well-suited for protecting API services built with Encore. This guide will show you how to integrate Logto with your Encore application to add authentication and authorization capabilities. You can find the complete [Logto example](https://github.com/encoredev/examples/tree/main/logto-react-sdk) in our examples repo. ## Logto settings Before we begin integrating with Encore, you'll need to set up a few things in Logto: 1. Create an account at [Logto Cloud](https://cloud.logto.io) if you don't have one yet. 2. Create an API Resource in Logto Console, this represents your Encore API service - Go to "API Resources" in Logto Console and create a new API - Set a name and API identifier (e.g., `https://api.encoreapp.com`) - Note down the API identifier on the API resource details page as we'll need it later ![Logto API Resource](https://encore.dev/assets/docs/logto-api-resource.png) 3. Create an application for your frontend application - Go to "Applications" in Logto Console - Create a new application according to your frontend framework (We use React as an example, but you can create any Single-Page Application (SPA) or native app) - (Optional, we'll cover this later) Integrate Logto with your frontend application according to the guide in the Logto Console. - Note down the application ID and issuer URL on the Application details page as we'll need them later ![Logto application endpoints](https://encore.dev/assets/docs/logto-application-endpoints.png) ## Setup the auth handler Now let's implement the authentication in your Encore application. We'll use Encore's built-in [auth handler](https://encore.dev/docs/go/develop/auth) to validate Logto's JWT tokens. Add these two modules in your Encore application: ```shell $ go get github.com/golang-jwt/jwt/v5 $ go get github.com/MicahParks/keyfunc/v3 ``` Create `auth/auth.go` and add the following code: ```go -- auth/auth.go -- package auth import ( "context" "time" "encore.dev/beta/auth" "encore.dev/beta/errs" "encore.dev/config" "github.com/MicahParks/keyfunc/v3" "github.com/golang-jwt/jwt/v5" ) // Configuration variables for authentication type LogtoAuthConfig struct { // The issuer URL Issuer config.String // URL to fetch JSON Web Key Set (JWKS) JwksUri config.String // Expected audience for the JWT ApiResourceIndicator config.String // Expected client ID in the token claims ClientId config.String } var authConfig *LogtoAuthConfig = config.Load[*LogtoAuthConfig]() // RequiredClaims defines the expected structure of JWT claims // Extends the standard JWT claims with a custom ClientID field type RequiredClaims struct { ClientID string `json:"client_id"` jwt.RegisteredClaims } // AuthHandler validates JWT tokens and extracts the user ID // Implements Encore's authentication handler interface // //encore:authhandler func AuthHandler(ctx context.Context, token string) (auth.UID, error) { // Fetch and parse the JWKS (JSON Web Key Set) from the identity provider jwks, err := keyfunc.NewDefaultCtx(ctx, []string{authConfig.JwksUri()}) if err != nil { return "", &errs.Error{ Code: errs.Internal, Message: "failed to fetch JWKS", } } // Parse and validate the JWT token with required claims and validation options parsedToken, err := jwt.ParseWithClaims( token, &RequiredClaims{}, jwks.Keyfunc, // Expect the token to be intended for this API resource jwt.WithAudience(authConfig.ApiResourceIndicator()), // Expect the token to be issued by this issuer jwt.WithIssuer(authConfig.Issuer()), // Allow some leeway for clock skew jwt.WithLeeway(time.Minute*10), ) // Check if there were any errors during token parsing if err != nil { return "", &errs.Error{ Code: errs.Unauthenticated, Message: "invalid token", } } // Verify that the client ID in the token matches the expected client ID if parsedToken.Claims.(*RequiredClaims).ClientID != authConfig.ClientId() { return "", &errs.Error{ Code: errs.Unauthenticated, Message: "invalid token", } } // Extract the user ID (subject) from the token claims userId, err := parsedToken.Claims.GetSubject() if err != nil { return "", &errs.Error{ Code: errs.Unauthenticated, Message: "invalid token", } } // Return the user ID as an Encore auth.UID return auth.UID(userId), nil } ``` Create a [configuration file](https://encore.dev/docs/go/develop/config) in the auth service and name it `auth-config.cue`. Add the following: ```cue -- auth/auth-config.cue -- Issuer: "" JwksUri: "/jwks" ApiResourceIndicator: "" ClientId: "" ``` Replace the values with the ones you noted down from your Logto settings: - ``: The issuer URL from your Logto application endpoints (e.g., `https://your-tenant.logto.app`) - ``: The API identifier you set when creating the API resource (e.g., `https://api.encoreapp.com`) - ``: The application ID from your Logto application details page For example, your `auth-config.cue` might look like: ```cue -- auth/auth-config.cue -- Issuer: "https://your-tenant.logto.app" JwksUri: "https://your-tenant.logto.app/jwks" ApiResourceIndicator: "https://api.encoreapp.com" ClientId: "2gadf3mp0zotlq8j1k5x" ``` And then, you can use this auth handler to protect your API endpoints: ```go package api import ( "context" "fmt" "encore.dev/beta/auth" "encore.dev/beta/errs" ) //encore:api auth path=/api/hello func Api(ctx context.Context) (*Response, error) { userId, hasUserId := auth.UserID() if !hasUserId { return nil, &errs.Error{ Code: errs.Internal, Message: "User ID not found", } } msg := fmt.Sprintf("Hello, %s!", userId) return &Response{Message: msg}, nil } type Response struct { Message string } ``` ## Frontend We've completed our work in the Encore API service. Now we need to integrate Logto with our frontend application. You can choose the framework you are using in the [Logto Quick start](https://docs.logto.io/quick-starts) page to integrate Logto with your frontend application. In this guide we use React as an example. Check out the [Add authentication to your React application](https://docs.logto.io/quick-starts/react) guide to learn how to integrate Logto with your React application. In this example, you only need to complete up to the Integration section. After that, we'll demonstrate how the frontend application can obtain an access token from Logto to access the Encore API. First, update your `LogtoConfig` by adding the API resource used in your Encore app to the `resources` field. This tells Logto that we will be requesting access tokens for this API resource (Encore API). ```ts import { LogtoConfig } from '@logto/react'; const config: LogtoConfig = { // ...other configs resources: [''], }; ``` After updating the `LogtoConfig`, if a user is already signed in, they need to sign out and sign in again for the new `LogtoConfig` settings to take effect. Once the user is logged in, you can use the `getAccessToken` method provided by the Logto React SDK to obtain an access token for accessing specific API resources. For example, to access the Encore API, we use `https://api.encoreapp.com` as the API resource identifier. Then, add this access token to the request headers as the `Authorization` field in subsequent requests. ```ts const { getAccessToken } = useLogto(); const accessToken = await getAccessToken(''); // Add this access token to the request headers as the 'Authorization' field in subsequent requests fetch('/hello', { headers: { Authorization: `Bearer ${accessToken}`, }, }); ``` Here's the key frontend code: ```tsx -- config/logto.tsx -- import { LogtoConfig } from '@logto/react' export const config: LogtoConfig = { endpoint: '', appId: '', resources: [''], } export const appConfig = { apiResourceIndicator: '', signInRedirectUri: '', signOutRedirectUri: '', } export const encoreApiEndpoint = '' -- pages/ProtectedResource.tsx -- import { useLogto } from "@logto/react"; import { useState } from "react"; import { Navigate } from "react-router-dom"; import { appConfig, encoreApiEndpoint } from "../config/logto"; export function ProtectedResource() { const { isAuthenticated, getAccessToken } = useLogto(); const [message, setMessage] = useState(""); const [isLoading, setIsLoading] = useState(false); const [error, setError] = useState(""); const fetchProtectedResource = async () => { setIsLoading(true); setError(""); try { const accessToken = await getAccessToken(appConfig.apiResourceIndicator); const response = await fetch(`${encoreApiEndpoint}/api/hello`, { headers: { Authorization: `Bearer ${accessToken}`, }, }); if (!response.ok) { throw new Error(`HTTP error! status: ${response.status}`); } const data = await response.json(); setMessage(JSON.stringify(data)); } catch (error) { console.error("Error fetching protected resource:", error); setError("Failed to fetch protected resource. Please try again."); } finally { setIsLoading(false); } }; if (!isAuthenticated) { return ; } return (

Protected Resource

{message && !error && (

Response from Protected API

{message}
)} {error &&
{error}
}
); } ``` That's it, you've successfully integrated Logto with your Encore application. You can find the complete example code [here](https://github.com/encoredev/examples/tree/main/logto-react-sdk). ## Explore more If you want to use more Logto features, you can refer to the following links for more information: - Combine Logto's [Custom token claims](https://docs.logto.io/developers/custom-token-claims) to set [custom user data](https://encore.dev/docs/go/develop/auth#with-custom-user-data) in the auth handler - Use [Logto RBAC features](https://docs.logto.io/authorization/role-based-access-control) to add authorization support to your application. The React integration tutorial also demonstrates how to add `scope` information to your Access token (note that you need to sign in again after updating Logto config) --- Source: https://encore.dev/docs/go/develop/api-docs Section: Building (Encore.go) # API Documentation > Write doc comments that are automatically surfaced across your app Encore parses doc comments from your Go source code and surfaces them in the [Service Catalog](https://encore.dev/docs/service-catalog), [generated API clients](https://encore.dev/docs/go/cli/client-generation), and OpenAPI specs. This means your documentation stays in your code and is always up to date. ## Services A service's documentation comes from the Go package doc comment — the comment block immediately before the `package` declaration. ```go // Payments handles payment processing, billing, // and subscription management. package payments ``` Service docs appear in the Service Catalog and in generated API clients. ### Service structs If you use a [service struct](https://encore.dev/docs/go/primitives/service-structs), the doc comment on the struct type is also extracted. ```go // PaymentService manages payment processing state and dependencies. // //encore:service type PaymentService struct { // ... } ``` ## API endpoints Add a comment block above the `//encore:api` directive to document an endpoint. ```go // Charge charges the given payment method. // It is idempotent; calling it multiple times // with the same idempotency key has no additional effect. // //encore:api auth func Charge(ctx context.Context, params *ChargeParams) (*ChargeResponse, error) { // ... } ``` In the OpenAPI spec, the first line becomes the `summary` and the remaining lines become the `description`. ## Auth handlers ```go // AuthenticateRequest validates the API key provided in the Authorization header // and returns the authenticated user's information. // //encore:authhandler func AuthenticateRequest(ctx context.Context, token string) (auth.UID, *UserData, error) { // ... } ``` ## Middleware ```go // LogRequests logs all incoming requests and their duration. // //encore:middleware target=all func LogRequests(req middleware.Request, next middleware.Next) middleware.Response { // ... } ``` ## Types Document types with a comment above the type declaration. ```go // ChargeParams defines the parameters for the Charge endpoint. type ChargeParams struct { // PaymentMethodID is the identifier of the payment method to charge. PaymentMethodID string // Amount is the amount to charge, in the smallest currency unit (e.g. cents). Amount int Currency string // ISO 4217 currency code } ``` Both the type-level and field-level comments are surfaced in generated clients and OpenAPI schemas. ## Infrastructure resources Doc comments are also supported on infrastructure resource declarations. ```go // TransactionDB stores payment transactions and related records. var TransactionDB = sqldb.NewDatabase("transactions", sqldb.DatabaseConfig{ Migrations: "./migrations", }) // PaymentEvents publishes events when payment states change. var PaymentEvents = pubsub.NewTopic[*PaymentEvent]("payment-events", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) // ProcessPaymentEvent handles incoming payment events. var _ = pubsub.NewSubscription(PaymentEvents, "process-payment", pubsub.SubscriptionConfig[*PaymentEvent]{ Handler: handlePaymentEvent, }) // DailySettlement runs the settlement process every day at midnight. var DailySettlement = cron.NewJob("daily-settlement", cron.JobConfig{ Every: 24 * cron.Hour, Endpoint: Settle, }) // PaymentCache caches frequently accessed payment records. var PaymentCache = cache.NewCluster("payment-cache", cache.ClusterConfig{ EvictionPolicy: cache.AllKeysLRU, }) // PaymentByID caches payments by their unique identifier. var PaymentByID = cache.NewStringKeyspace[Payment](PaymentCache, cache.KeyspaceConfig{ KeyPattern: "payment/:id", }) // Receipts stores generated receipt PDFs. var Receipts = objects.NewBucket("receipts", objects.BucketConfig{}) // ChargeAmount tracks the total amount charged per currency. var ChargeAmount = metrics.NewCounterGroup[ChargeLabels, uint64]("charge_amount", metrics.CounterConfig{}) type ChargeLabels struct { // Currency is the ISO 4217 currency code. Currency string } ``` These docs appear in the Service Catalog and are available through the [MCP server](https://encore.dev/docs/ai-integration#connect-the-local-mcp-server) for AI-assisted development. ## Where docs appear | Where | What's included | |---|---| | [Service Catalog](https://encore.dev/docs/service-catalog) | Services, endpoints, types, fields | | [Generated clients](https://encore.dev/docs/go/cli/client-generation) | Services, endpoints, types, fields | | OpenAPI spec | Endpoints (summary + description), types, fields | | [MCP server](https://encore.dev/docs/ai-integration#connect-the-local-mcp-server) | All resource types | --- Source: https://encore.dev/docs/go/cli/client-generation Section: Building (Encore.go) # Client Library Generation > Stop writing the same types everywhere Encore makes it simple to write scalable distributed backends by allowing you to make function calls that Encore translates into RPC calls. Encore also generates API clients with interfaces that look like the original Go functions, with the same parameters and response signature as the server. The generated clients are single files that use only the standard functionality of the target language, with full type safety. This allow anyone to look at the generated client and understand exactly how it works. The structure of the generated code varies by language, to ensure it's idiomatic and easy to use, but always includes all publicly accessible endpoints, data structures, and documentation strings. Encore currently supports generating the following clients: - **Go** - Using `net/http` for the underlying HTTP transport. - **TypeScript** - Using the browser `fetch` API for the underlying HTTP client. - **JavaScript** - Using the browser `fetch` API for the underlying HTTP client. - **OpenAPI** - Using the OpenAPI Specification's language-agnostic interface to HTTP APIs. (Experimental) If there's a language you think should be added, please submit a pull request or create a feature request on [GitHub](https://github.com/encoredev/encore/issues/new), or [reach out on Discord](https://encore.dev/discord). > **Important** > > If you ship the generated client to end customers, keep in mind that old clients will continue to be used after you make changes. To prevent issues with the generated clients, avoid making breaking changes in APIs that your clients access.
## Generating a Client To generate a client, use the `encore gen client` command. It generates a type-safe client using the most recent API metadata running in a particular environment for the given Encore application. For example: ```shell # Generate a TypeScript client for calling the hello-a8bc application based on the primary environment encore gen client hello-a8bc --output=./client.ts # Generate a Go client for the hello-a8bc application based on the locally running code encore gen client hello-a8bc --output=./client.go --env=local # Generate an OpenAPI client for the hello-a8bc application based on the primary environment encore gen client hello-a8bc --lang=openapi --output=./openapi.json ``` ### Environment Selection By default, `encore gen client` generates the client based on the version of your application currently running in your local environment. You can change this using the `--env` flag and specifying the environment name. > **Note** > > The generated client can be used with any environment, not just the one it was generated for. However, the APIs, data structures > and marshalling logic will be based on whatever is present and running in that environment at the point in time the client is generated. ### Service filtering By default `encore gen client` outputs code for all services with at least one publicly accessible (or authenticated) API. You can narrow down this set of services by specifying the `--services` (or `-s`) flag. It takes a comma-separated list of service names. For example, to generate a typescript client for the `email` and `users` services, run: ```shell encore gen client --services=email,users -o client.ts ``` ### Output Mode By default the client's code will be output to stdout, allowing you to pipe it into your clipboard, or another tool. However, using `--output` you can specify a file location to write the client to. If output is specified, you do not need to specify the language as Encore will detect the language based on the file extension. ### Example Script You could combine this into a `package.json` file for your Typescript frontend, to allow you to run `npm run gen` in that project to update the client to match the code running in your staging environment. ```json -- package.json -- { "scripts": { // ... "gen": "encore gen client hello-a8bc --output=./client.ts --env=staging" // ... } } ``` ## Using the Client The generated client has all the data structures required as parameters or returned as response values as needed by any of the public or authenticated API's of your Encore application. Each service is exposed as object on the client, with each public or authenticated API exposed as a function on those objects. For instance, if you had a service called `email` with a function `Send`, on the generated client you would call this using; `client.email.Send(...)`. For more tips and examples of using a generated JavaScript/Typescript client, see the [Integrate with a web frontend](https://encore.dev/docs/go/how-to/integrate-frontend#generating-a-request-client) docs. ### Creating an instance When constructing a client, you need to pass a `BaseURL` as the first parameter; this is the URL at which the API can be accessed. The client provides two helpers: - `Local` - This is a constant provided, which will always point at your locally running instance environment. - `Environment("name")` - This is a function which allows you to specify an environment by name However, BaseURL is a string, so if the two helpers do not provide enough flexibility you can pass any valid URL to be used as the BaseURL. ### Authentication If your application has any API's which require [authentication](https://encore.dev/docs/go/develop/auth), then additional options will generated into the client, which can be used when constructing the client. Just like with API's schemas, the data type required by your application's `auth handler` will be part of the client library, allowing you to set it in two ways: If your credentials won't change during the lifetime of the client, simply passing the authentication data to the client through the `WithAuth` (Go) or `auth` (TypeScript) options. However, if the authentication credentials can change, you can also pass a function which will be called before each request and can return a new instance of the authentication data structure or return the existing instance. ### HTTP Client Override If required, you can override the underlying HTTP implementation with your own implementation. This is useful if you want to perform logging of the requests being made, or route the traffic over a secured tunnel such as a VPN. In Go this can be configured using the `WithHTTPClient` option. You are required to provide an implementation of the `HTTPDoer` interface, which the [http.Client](https://pkg.go.dev/net/http#Client) implements. For TypeScript clients, this can be configured using the `fetcher` option and must conform to the same prototype as the browsers inbuilt [fetch API](https://developer.mozilla.org/en-US/docs/Web/API/fetch). ### Structured Errors Errors created or wrapped using Encore's [`errs package`](https://encore.dev/docs/go/primitives/api-errors) will be returned to the client and deserialized as an `APIError`, allowing the client to perform adaptive error handling based on the type of error returned. You can perform a type check on errors caused by calling an API to see if it is an `APIError`, and once cast as an `APIError` you can access the `Code`, `Message` and `Details` fields. For TypeScript Encore generates a `isAPIError` type guard which can be used. The `Code` field is an enum with all the possible values generated in the library, alone with description of when we would expect them to be returned by your API. See the [errors documentation](https://encore.dev/docs/go/primitives/api-errors#error-codes) for an online reference of this list. ## Example CLI Tool For instance, we could build a simple CLI application to use our [url shortener](https://encore.dev/docs/go/tutorials/rest-api), and handle any structured errors in a way which makes sense for that error code. ```go package main import ( "context" "fmt" "os" "time" "shorten_cli/client" ) func main() { // Create a new client with the default BaseURL client, err := client.New( client.Environment("production"), client.WithAuth(os.Getenv("SHORTEN_API_KEY")), ) if err != nil { panic(err) } // Timeout if the request takes more than 5 seconds ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() // Call the Shorten function in the URL service resp, err := client.Url.Shorten( ctx, client.UrlShortenParams{ URL: os.Args[1] }, ) if err != nil { // Check the error returned if err, ok := err.(*client.APIError); ok { switch err.Code { case client.ErrUnauthenticated: fmt.Println("SHORTEN_API_KEY was invalid, please check your environment") os.Exit(1) case client.ErrAlreadyExists: fmt.Println("The URL you provided was already shortened") os.Exit(0) } } panic(err) // if here then something has gone wrong in an unexpected way } fmt.Printf("https://short.encr.app/%s", resp.ID) } ``` --- Source: https://encore.dev/docs/go/how-to/integrate-frontend Section: Building (Encore.go) # Integrate with a web frontend > Keep using your favorite frontend hosting provider Encore is not opinionated about where you host your frontend, pick the platform that suits your situation best. If your frontend and backend use different domains, often the case when using PR preview environments for your frontend, you may need to [configure CORS](#handling-cors). Take a look at our [React starter template](https://encore.dev/templates/react) for an example of deploying a frontend to [Vercel](https://vercel.com/) or the [Meeting Notes tutorial](https://encore.dev/docs/tutorials/uptime) deployed to [GitHub Pages](https://pages.github.com/). ## Generating a request client Encore is able to generate frontend request clients (TypeScript or JavaScript). This lets you to keep the request/response types in sync without manual work and assists you in calling the APIs. Generate a client by running: ```bash $ encore gen client --output=./src/client.ts --env= ``` Adding this as a script to your `package.json` is often a good idea to be able to run it whenever a change is made to your Encore API: ```json -- package.json -- { ... "scripts": { ... "generate-client:staging": "encore gen client --output=./src/client.ts --env=staging", "generate-client:local": "encore gen client --output=./src/client.ts --env=local" } } ``` After that you are ready to use the request client in your code. Here is an example from the [Meeting Notes tutorial](https://encore.dev/docs/tutorials/meeting-notes) for calling the `GetNote` endpoint on the `note` service in order to retrieve a specific meeting note (which has the properties `id`, `cover_url` & `text`): ```ts import Client, { Environment, Local } from "src/client.ts"; // Making request to locally running backend... const client = new Client(Local); // or to a specific deployed environment const client = new Client(Environment("staging")); // Calling APIs as typesafe functions 🌟 const response = await client.note.GetNote("note-uuid"); console.log(response.id); console.log(response.cover_url); console.log(response.text); ``` See more in the [client generation docs](https://encore.dev/docs/go/cli/client-generation). ### Asynchronous state management When building something a bit more complex, you will likely need to deal with caching, refetching, and data going stale. [TanStack Query](https://tanstack.com/query/latest) is a popular library that was built to solve exactly these problems and works well with the Encore request client. Here is a simple example of using an Encore request client together with TanStack Query: ```ts import { useQuery, useMutation, useQueryClient, QueryClient, QueryClientProvider, } from '@tanstack/react-query' import Client, { todo } from '../encore-client' // Create a Encore client const encoreClient = new Client(window.location.origin); // Create a react-query client const queryClient = new QueryClient() function App() { return ( // Provide the client to your App ) } function Todos() { // Access the client const queryClient = useQueryClient() // Queries const query = useQuery({ queryKey: ['todos'], queryFn: () => encoreClient.todo.List() }) // Mutations const mutation = useMutation({ mutationFn: (params: todo.AddParams) => encoreClient.todo.Add(params), onSuccess: () => { // Invalidate and refetch queryClient.invalidateQueries({ queryKey: ['todos'] }) }, }) return (
    {query.data?.map((todo) => (
  • {todo.title}
  • ))}
) } render(, document.getElementById('root')) ``` This example assumes that we have a `todo` service with a `List` and `Add` endpoint. When adding the new todo, TanStack Query will automatically invalidate the `todos` query and refetch it. For a real-world example, take a look at the [Uptime Monitoring](https://github.com/encoredev/examples/tree/main/uptime) app which also makes use of TanStack Query's `refetchInterval` option for polling the backend. ### Testing When unit testing a component that interacts with your Encore API you can mock methods on the request client to return a value suitable for the test. This makes your test URL agnostic because you are not intercepting specific requests on the fetch layer. You also get type errors in your tests if the request client gets updated. Here is an example from the [Uptime Monitoring Starter](https://github.com/encoredev/examples/tree/main/uptime) where we are mocking a GET request method and spying on a POST request method: ```ts import { render, waitForElementToBeRemoved } from "@testing-library/react"; import App from "./App"; import { site } from "./client"; import { userEvent } from "@testing-library/user-event"; describe("App", () => { beforeEach(() => { // Return mocked data from the List (GET) endpoint jest .spyOn(site.ServiceClient.prototype, "List") .mockReturnValue(Promise.resolve({ sites: [{ id: 1, url: "test.dev" }] })); // Spy on the Add (POST) endpoint jest.spyOn(site.ServiceClient.prototype, "Add"); }); it("render sites", async () => { render(); await waitForElementToBeRemoved(() => screen.queryByText("Loading...")); // Verify that the List endpoint has been called expect(site.ServiceClient.prototype.List).toBeCalledTimes(1); // Verify that the sites are rendered with our mocked data screen.getAllByText("test.dev"); }); it("add site", async () => { render(); await waitForElementToBeRemoved(() => screen.queryByText("Loading...")); // Interact with the page and add 'another.com' await userEvent.click(screen.getByText("Add website")); await userEvent.type( screen.getByPlaceholderText("google.com"), "another.com", ); await userEvent.click(screen.getByText("Save")); // Verify that the Add endpoint has been called with the correct parameters expect(site.ServiceClient.prototype.Add).toHaveBeenCalledWith({ url: "another.com", }); }); }) ``` > **Note** > > In the example above we need to mock the `List` method on `site.ServiceClient.prototype` because the request client has not > yet been initialized when we're creating the mock. If you have access to the instance of the request client in your test > (which could be the case if you are passing the client around in your components) you can instead do `jest.spyOn(client.site, "List")` > and `expect(client.site.List).toHaveBeenCalled()` which would give you the same result. More examples of tests can be found in the [Uptime Monitoring Starter repo](https://github.com/encoredev/examples/tree/main/uptime). ## Monorepo or Multi repo Encore is not opinionated about where your frontend lives, pick the approach that fits your application best. If you use a monorepo then it is often a good idea to place your backend and frontend in separate folders. There are two approaches to moving your Encore backend to a subfolder: 1. Place your microservices together with the `encore.app` file in a subfolder. When moving `encore.app` to a subfolder you will need to configure the "Root Directory" in app settings in the [Encore platform dashboard](https://app.encore.dev). 2. Place your microservices in a subfolder and keep the `encore.app` in the repo root directory. No configuration change is needed, but you will need to update the import paths if your services are calling each other. ## REST vs. GraphQL Encore allows for building backends using both REST and GraphQL, you should pick the approach that suits your use case best. Take a look at the [GraphQL tutorial](https://encore.dev/docs/go/tutorials/graphql) for an example of building a GraphQL backend with Encore. ## Hosting a frontend on Encore for development Encore is primarily designed for backend development and does not (at the moment) support building or testing frontends in the deploy pipeline. For production use, we recommend that you deploy your frontend using Vercel, Netlify, or a similar service. For development purposes, you can create a `raw` endpoint that serves static frontend assets. It would look something like the example below (taken from the [Uptime Monitoring tutorial](https://encore.dev/docs/tutorials/uptime)), but keep in mind that you need to have the compiled frontend assets under version control (`dist` folder in the example below). ```go package frontend import ( "embed" "io/fs" "net/http" ) var ( //go:embed dist dist embed.FS assets, _ = fs.Sub(dist, "dist") handler = http.StripPrefix("/frontend/", http.FileServer(http.FS(assets))) ) //encore:api public raw path=/frontend/*path func Serve(w http.ResponseWriter, req *http.Request) { handler.ServeHTTP(w, req) } ``` ## Handling CORS If you are running into CORS issues when calling your Encore API from your frontend you may need to specify which origins are allowed to access your API (via browsers). Do this by specifying the `global_cors` key in the `encore.app` file, which has the following structure: ```json -- encore.app -- global_cors: { // allow_origins_without_credentials specifies the allowed origins for requests // that don't include credentials. If nil it defaults to allowing all domains // (equivalent to ["*"]). "allow_origins_without_credentials": [ "" ], // allow_origins_with_credentials specifies the allowed origins for requests // that include credentials. If a request is made from an Origin in this list // Encore responds with Access-Control-Allow-Origin: . // // The URLs in this list may include wildcards (e.g. "https://*.example.com" // or "https://*-myapp.example.com"). "allow_origins_with_credentials": [ "" ] } ``` See more in the [CORS docs](https://encore.dev/docs/cors). --- Source: https://encore.dev/docs/go/tutorials/rest-api Section: Building (Encore.go) # Building a REST API > Learn how to build a URL shortener with a REST API and PostgreSQL database In this tutorial you will create a REST API for a URL Shortener service. In a few short minutes, you'll learn how to: * Create REST APIs with Encore * Use PostgreSQL databases * Use the local development dashboard to test your app * Create and run tests This is the end result: _[Interactive editor showing the finished `urlShortener` app — open this page in a browser to see it.]_ > **Note** > > To make it easier to follow along, we've laid out a trail of croissants to guide your way. > Whenever you see a 🥐 it means there's something for you to do. ## 1. Create a service and endpoint If you haven't already, create a new application by running `encore app create` and select `Empty app` as the template. If this is the first time you're using Encore, you'll be asked if you wish to create a free account. This is needed when you want Encore to manage functionality like secrets and handle cloud deployments (which we'll use later on in the tutorial). Now let's create a new `url` service. 🥐 In your application's root folder, create a new folder `url` and create a new file `url.go` that looks like this: ```go -- url/url.go -- // Service url takes URLs, generates random short IDs, and stores the URLs in a database. package url import ( "context" "crypto/rand" "encoding/base64" ) type URL struct { ID string // short-form URL id URL string // complete URL, in long form } type ShortenParams struct { URL string // the URL to shorten } // Shorten shortens a URL. //encore:api public method=POST path=/url func Shorten(ctx context.Context, p *ShortenParams) (*URL, error) { id, err := generateID() if err != nil { return nil, err } return &URL{ID: id, URL: p.URL}, nil } // generateID generates a random short ID. func generateID() (string, error) { var data [6]byte // 6 bytes of entropy if _, err := rand.Read(data[:]); err != nil { return "", err } return base64.RawURLEncoding.EncodeToString(data[:]), nil } ``` This sets up the `POST /url` endpoint (see the `//encore:api` annotation on the `Shorten` function). 🥐 Let’s see if it works! Start your app by running `encore run`. You should see this: ```output Your API is running at: http://localhost:4000 Development Dashboard URL: http://localhost:9400 4:19PM TRC registered endpoint path=/url service=url endpoint=Shorten ``` 🥐 Next, call your endpoint from the Local Development Dashboard at [http://localhost:9400](http://localhost:9400) and view a trace of the response. It should look like this: You can also call it from the terminal: ```shell $ curl http://localhost:4000/url -d '{"URL": "https://encore.dev"}' ``` And you should see this: ```json { "ID": "5cJpBVRp", "URL": "https://encore.dev" } ``` It works! There’s just one problem... Right now, we’re not actually storing the URL anywhere. That means we can generate shortened IDs but there’s no way to get back to the original URL! We need to store a mapping from the short ID to the complete URL. ## 2. Save URLs in a database Fortunately, Encore makes it really easy to set up a PostgreSQL database to store our data. To do so, we first define a **database schema**, in the form of a migration file. 🥐 Create a new folder named `migrations` inside the `url` folder. Then, inside the `migrations` folder, create an initial database migration file named `1_create_tables.up.sql`. The file name format is important (it must start with `1_` and end in `.up.sql`). 🥐 Add the following contents to the file: ```sql -- url/migrations/1_create_tables.up.sql -- CREATE TABLE url ( id TEXT PRIMARY KEY, original_url TEXT NOT NULL ); ``` 🥐 Next, go back to the `url/url.go` file and import the `encore.dev/storage/sqldb` package by modifying the import statement to become: ```go HL url/url.go 5:5 -- url/url.go -- import ( "context" "crypto/rand" "encoding/base64" "encore.dev/storage/sqldb" ) ``` 🥐 Then let's define our database object by adding the following to `url/url.go`: ```go -- url/url.go -- // Define a database named 'url', using the database // migrations in the "./migrations" folder. var db = sqldb.NewDatabase("url", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` 🥐 Now, to insert data into our database, let’s create a helper function `insert`: ```go -- url/url.go -- // insert inserts a URL into the database. func insert(ctx context.Context, id, url string) error { _, err := db.Exec(ctx, ` INSERT INTO url (id, original_url) VALUES ($1, $2) `, id, url) return err } ``` 🥐 Lastly, we can update our `Shorten` function to insert into the database: ```go -- url/url.go -- func Shorten(ctx context.Context, p *ShortenParams) (*URL, error) { id, err := generateID() if err != nil { return nil, err } else if err := insert(ctx, id, p.URL); err != nil { return nil, err } return &URL{ID: id, URL: p.URL}, nil } ``` > **Note** > > Before running your application, make sure you have [Docker](https://www.docker.com) installed and running. It's required to locally run Encore applications with databases. 🥐 Next, start the application again with `encore run` and Encore automatically sets up your database. (In case your application won't run, check the [databases troubleshooting guide](https://encore.dev/docs/go/primitives/databases/troubleshooting).) You can verify that the database was created by opening the **Infra** tab in the local development dashboard at [localhost:9400](http://localhost:9400), which should something like this: ![Infra tab in local development dashboard](https://encore.dev/assets/docs/infra_tab.png) 🥐 Now let's call the API again from the local development dashboard, or from the terminal: ```shell $ curl http://localhost:4000/url -d '{"URL": "https://encore.dev"}' ``` 🥐 Finally, let's verify that it was saved in the database. You can do this by checking the trace in the local development dashboard, or you can run `encore db shell url` from the app root directory and inputting `select * from url;`: ```shell $ encore db shell url psql (13.1, server 11.12) Type "help" for help. url=# select * from url; id | original_url ----------+-------------------- zr6RmZc4 | https://encore.dev (1 row) ``` That was easy! ## 3. Add endpoint to retrieve URLs To complete our URL shortener API, let’s add the endpoint to retrieve a URL given its short id. 🥐 Add this endpoint to `url/url.go`: ```go -- url/url.go -- // Get retrieves the original URL for the id. //encore:api public method=GET path=/url/:id func Get(ctx context.Context, id string) (*URL, error) { u := &URL{ID: id} err := db.QueryRow(ctx, ` SELECT original_url FROM url WHERE id = $1 `, id).Scan(&u.URL) return u, err } ``` Encore uses the `path=/url/:id` syntax to represent a path with a parameter. The `id` name corresponds to the parameter name in the function signature. In this case it is of type `string`, but you can also use other built-in types like `int` or `bool` if you want to restrict the values. 🥐 We can make sure it works by reviewing the endpoint in the Service Catalog in the local development dashboard, where we can call it using the `id` you got in the previous step: You can also call it directly from the terminal: ```shell $ curl http://localhost:4000/url/zr6RmZc4 ``` You should now see this: ```json { "ID": "zr6RmZc4", "URL": "https://encore.dev" } ``` It works! That's how you build REST APIs and use PostgreSQL databases in Encore. ## 4. Add a test Before deployment, it is good practice to have tests to assure that the service works properly. Such tests including database access are easy to write. We've prepared a test to check that the whole cycle of shortening the URL, storing and then retrieving the original URL works. 🥐 Save this in a separate file `url/url_test.go`. ```go -- url/url_test.go -- package url import ( "context" "testing" ) // TestShortenAndRetrieve - test that the shortened URL is stored and retrieved from database. func TestShortenAndRetrieve(t *testing.T) { testURL := "https://github.com/encoredev/encore" sp := ShortenParams{URL: testURL} resp, err := Shorten(context.Background(), &sp) if err != nil { t.Fatal(err) } wantURL := testURL if resp.URL != wantURL { t.Errorf("got %q, want %q", resp.URL, wantURL) } firstURL := resp gotURL, err := Get(context.Background(), firstURL.ID) if err != nil { t.Fatal(err) } if *gotURL != *firstURL { t.Errorf("got %v, want %v", *gotURL, *firstURL) } } ``` 🥐 Now run `encore test ./...` to verify that it's working. If you use the local development dashboard ([localhost:9400](http://localhost:9400)), you can even see traces for tests. ## 5. Deploy ### Self-hosting Encore supports building Docker images directly from the CLI, which can then be self-hosted on your own infrastructure of choice. If your app is using infrastructure resources, such as SQL databases, Pub/Sub, or metrics, you will need to configure your Docker image with the necessary configuration. Our URL shortener makes use of a PostgreSQL database, so we'll need to supply a [runtime configuration](https://encore.dev/docs/self-host/configure-infra) so that our app knows how to connect to the database in the cloud. 🥐 Create a new file `infra-config.json` in the root of your project with the following contents: ```json -- infra-config.json -- { "$schema": "https://encore.dev/schemas/infra.schema.json", "sql_servers": [ { "host": "my-db-host:5432", "databases": { "url": { "username": "my-db-owner", "password": {"$env": "DB_PASSWORD"} } } } ] } ``` The values in this configuration are just examples, you will need to replace them with the correct values for your database. 🥐 Build a Docker image by running `encore build docker --config infra-config.json url-shortener:v1.0`. This will compile your application using the host machine and then produce a Docker image containing the compiled application. 🥐 Upload the Docker image to the cloud provider of your choice and run it. ### The Encore platform The Encore platform provides automated infrastructure and DevOps. Deploy to a free Encore Cloud development environment, or connect your own AWS or GCP account for production. ### Create account Before deploying with the Encore platform, your app needs to be linked to a free Encore account. If you created the app while logged in, it's already linked, and you can move on to the next step. Otherwise, sign up or log in and register the app by following steps 1 to 3 of [Already have a local app?](https://encore.dev/docs/platform/deploy/deploying#already-have-a-local-app), then continue below. ### Commit changes The final step before you deploy is to commit all changes to the project repo. 🥐 Commit the new files to the project's git repo and trigger a deploy to a free Encore Cloud development environment by running: ```shell $ git add -A . $ git commit -m 'Initial commit' $ git push encore ``` Encore will now build and test your app, provision the needed infrastructure, and deploy your application to the cloud. After triggering the deployment, you will see a URL where you can view its progress in the [Encore platform dashboard](https://app.encore.dev). It will look something like: `https://app.encore.dev/$APP_ID/envs/staging/deploys/...` From there you can also see metrics, traces, and connect your own AWS or GCP account to use for production deployment. *Now you have a fully fledged backend running in the cloud, well done!* ### Celebrate with fireworks Now that your app is running in the cloud, let's celebrate with some fireworks: 🥐 In the Cloud Dashboard, open the Command Menu by pressing **Cmd + K** (Mac) or **Ctrl + K** (Windows/Linux). _From here you can easily access all Cloud Dashboard features and for example jump straight to specific services in the Service Catalog or view Traces for specific endpoints._ 🥐 Type `fireworks` in the Command Menu and press enter. Sit back and enjoy the show! ![Fireworks](https://encore.dev/assets/docs/fireworks.jpg) 🥐 A great next step is to [integrate with GitHub](https://encore.dev/docs/platform/integrations/github). Once you've linked with GitHub, Encore will automatically start building and running tests against your Pull Requests. ## What's next Now that you know how to build a backend with a database, you're ready to let your creativity flow and begin building your next great idea! We're excited to hear what you're going to build with Encore, join the pioneering developer community on [Discord](https://encore.dev/discord) and share your story. --- Source: https://encore.dev/docs/go/tutorials/graphql Section: Building (Encore.go) # Building a GraphQL API > Learn how to build a GraphQL API using Encore.go Encore has great support for GraphQL with its type-safe approach to building APIs. Encore's automatic tracing also makes it easy to find and fix performance issues that often arise in GraphQL APIs (like the [N+1 problem](https://hygraph.com/blog/graphql-n-1-problem)). The best way to use GraphQL with Encore is using [gqlgen](https://gqlgen.com/), which has similar goals as Encore (type-safe APIs, minimal boilerplate, code generation, etc). The final code will look like this: _[Interactive editor showing the finished `graphql` app — open this page in a browser to see it.]_ ## 1. Create your Encore application This tutorial uses the [REST API](https://encore.dev/docs/go/tutorials/rest-api) tutorial as a starting point. You can either follow that tutorial first, or you can create a new Encore application using the `url-shortener` template by running: ```shell $ encore app create --example=url-shortener ``` ## 2. Initialize gqlgen To get started, initialize gqlgen by creating a `tools.go` file in the application root: ```go -- tools.go -- //go:build tools package tools import ( _ "github.com/99designs/gqlgen" _ "github.com/99designs/gqlgen/graphql/introspection" ) ``` Then run `go mod tidy` to download the dependencies. Next, create a `gqlgen.yml` file in the application root containing: ``` -- gqlgen.yml -- # Where are all the schema files located? globs are supported eg src/**/*.graphqls schema: - graphql/*.graphqls # Where should the generated server code go? exec: filename: graphql/generated/generated.go package: generated # Where should any generated models go? model: filename: graphql/model/models_gen.go package: model # Where should the resolver implementations go? resolver: layout: follow-schema dir: graphql package: graphql # gqlgen will search for any type names in the schema in these go packages # if they match it will use them, otherwise it will generate them. autobind: - "encore.app/url" # This section declares type mapping between the GraphQL and go type systems # # The first line in each type will be used as defaults for resolver arguments and # modelgen, the others will be allowed when binding to fields. Configure them to # your liking models: ID: model: - github.com/99designs/gqlgen/graphql.ID - github.com/99designs/gqlgen/graphql.Int - github.com/99designs/gqlgen/graphql.Int64 - github.com/99designs/gqlgen/graphql.Int32 Int: model: - github.com/99designs/gqlgen/graphql.Int - github.com/99designs/gqlgen/graphql.Int64 - github.com/99designs/gqlgen/graphql.Int32 ``` ## 3. Create Encore service Now it's time to create our Encore service that will provide the GraphQL API. First generate the gqlgen boilerplate: ```shell $ mkdir -p graphql/generated graphql/model $ echo "package model" > graphql/model/model.go $ go run github.com/99designs/gqlgen generate ``` This will create a bunch of files in the `graphql` directory. Next, create a `graphql/service.go` file containing: ```go -- graphql/service.go -- // Service graphql exposes a GraphQL API. package graphql import ( "net/http" "encore.app/graphql/generated" "encore.dev" "github.com/99designs/gqlgen/graphql/handler" "github.com/99designs/gqlgen/graphql/playground" ) //go:generate go run github.com/99designs/gqlgen generate //encore:service type Service struct { srv *handler.Server playground http.Handler } func initService() (*Service, error) { srv := handler.NewDefaultServer(generated.NewExecutableSchema(generated.Config{Resolvers: &Resolver{}})) pg := playground.Handler("GraphQL Playground", "/graphql") return &Service{srv: srv, playground: pg}, nil } //encore:api public raw path=/graphql func (s *Service) Query(w http.ResponseWriter, req *http.Request) { s.srv.ServeHTTP(w, req) } //encore:api public raw path=/graphql/playground func (s *Service) Playground(w http.ResponseWriter, req *http.Request) { // Disable playground in production if encore.Meta().Environment.Type == encore.EnvProduction { http.Error(w, "Playground disabled", http.StatusNotFound) return } s.playground.ServeHTTP(w, req) } ``` This creates an Encore service that exposes the `/graphql` and `/graphql/playground` endpoints. It also adds a `//go:generate` directive that lets you re-run the gqlgen code generation by running `go generate ./graphql`. ## 4. Add GraphQL schema Now it's time to define the GraphQL schema. Create a `graphql/schema.graphqls` file containing: ``` -- graphql/url.graphqls -- type Query { urls: [URL!]! get(id: ID!): URL! } type Mutation { shorten(input: String!): URL! } type URL { id: ID! # shortened id url: String! # full URL } ``` Then, re-run the code generation to generate the resolver stubs: ```shell $ go generate ./graphql ``` The stubs will be written to `graphql/url.resolvers.go` and will contain a bunch of unimplemented resolver methods that look something like this: ```go -- graphql/url.resolvers.go -- // Shorten is the resolver for the shorten field. func (r *mutationResolver) Shorten(ctx context.Context, input string) (*url.URL, error) { panic(fmt.Errorf("not implemented: Shorten - shorten")) } ``` ## 5. Implement resolvers Now, modify the resolvers to call the `url` service. Since the GraphQL API uses the same types (thanks to the `autobind` directive in `gqlgen.yml`) as the Encore API exposes, we can just call the endpoints directly. Implement the resolvers in `graphql/url.resolvers.go` like this: ```go -- graphql/url.resolvers.go -- // Shorten is the resolver for the shorten field. func (r *mutationResolver) Shorten(ctx context.Context, input string) (*url.URL, error) { return url.Shorten(ctx, &url.ShortenParams{URL: input}) } // Urls is the resolver for the urls field. func (r *queryResolver) Urls(ctx context.Context) ([]*url.URL, error) { resp, err := url.List(ctx) if err != nil { return nil, err } return resp.URLs, nil } // Get is the resolver for the get field. func (r *queryResolver) Get(ctx context.Context, id string) (*url.URL, error) { return url.Get(ctx, id) } ``` As you can see, the resolvers are just thin wrappers around the Encore API endpoints themselves. ## 6. Trying it out With that, the GraphQL API is done! Try it out by running `encore run` and opening up [the playground](http://localhost:4000/graphql/playground). Enter the query: ```graphql mutation { shorten(input: "https://encore.dev") { id } } ``` You should get back an id like `MnTWA8Jo`. Pass the id you got (it will be something different) to a `get` query: ```graphql query { get(id: "") { url } } ``` And you should get back `https://encore.dev`. ## 7. Deploy ### Self-hosting Encore supports building Docker images directly from the CLI, which can then be self-hosted on your own infrastructure of choice. If your app is using infrastructure resources, such as SQL databases, Pub/Sub, or metrics, you will need to supply a [runtime configuration](https://encore.dev/docs/self-host/configure-infra) file to your Docker image. This app uses the `url` service's database, so it needs one. 🥐 Build a Docker image by running `encore build docker --config infra-config.json graphql:v1.0`. This will compile your application using the host machine and then produce a Docker image containing the compiled application. 🥐 Upload the Docker image to the cloud provider of your choice and run it. ### The Encore platform The Encore platform provides automated infrastructure and DevOps. Deploy to a free Encore Cloud development environment, or connect your own AWS or GCP account for production. ### Create account Before deploying with the Encore platform, your app needs to be linked to a free Encore account. If you created the app while logged in, it's already linked, and you can move on to the next step. Otherwise, sign up or log in and register the app by following steps 1 to 3 of [Already have a local app?](https://encore.dev/docs/platform/deploy/deploying#already-have-a-local-app), then continue below. ### Commit changes The final step before you deploy is to commit all changes to the project repo. Push your changes and deploy your application to a free Encore Cloud development environment by running: ```shell $ git add -A . $ git commit -m 'Initial commit' $ git push encore ``` Encore will now build and test your app, provision the needed infrastructure, and deploy your application to the cloud. After triggering the deployment, you will see a URL where you can view its progress in the [Encore platform dashboard](https://app.encore.dev). It will look something like: `https://app.encore.dev/$APP_ID/envs/staging/deploys/...` From there you can also see metrics, traces, link your app to a GitHub repo to get automatic deploys on new commits, and connect your own AWS or GCP account to use for production deployment. ### Celebrate with fireworks Now that your app is running in the cloud, let's celebrate with some fireworks: 🥐 In the Cloud Dashboard, open the Command Menu by pressing **Cmd + K** (Mac) or **Ctrl + K** (Windows/Linux). _From here you can easily access all Cloud Dashboard features and for example jump straight to specific services in the Service Catalog or view Traces for specific endpoints._ 🥐 Type `fireworks` in the Command Menu and press enter. Sit back and enjoy the show! ![Fireworks](https://encore.dev/assets/docs/fireworks.jpg) ## Conclusion We've now built a GraphQL API gateway that forwards requests to the application's underlying Encore services in a type-safe way with minimal boilerplate. Note that the concepts discussed here are general and can be easily adapted to any GraphQL schema. Whenever you make a change to the schema or configuration, re-run `go generate ./graphql` to regenerate the GraphQL boilerplate. And for more information on how to use `gqlgen`, see the [gqlgen documentation](https://gqlgen.com/). --- Source: https://encore.dev/docs/go/how-to/cgo Section: Building (Encore.go) # Build with cgo Cgo is a feature of the Go compiler that enables Go programs to interface with libraries written in other languages using C bindings. By default, for improved portability Encore builds applications with cgo support disabled. To enable cgo for your application, add `"build": {"cgo_enabled": true}` to your `encore.app` file. For example: ```json -- encore.app -- { "id": "my-app-id", "build": { "cgo_enabled": true } } ``` With this setting Encore's build system will compile the application using an Ubuntu builder image with gcc pre-installed. ## Static linking To keep the resulting Docker images as minimal as possible, Encore compiles applications with static linking. This happens even with cgo enabled. As a result the cgo libraries you use must support static linking. In some cases, you may need to add additional linker flags to properly work with static linking of cgo libraries. See the [official cgo docs](https://pkg.go.dev/cmd/cgo) for more information on how to do this. --- Source: https://encore.dev/docs/py Section: Building (Encore.py) # Encore.py > An infrastructure SDK for Python Encore.py is the Python SDK for Encore. It lets you declare the infrastructure your application needs, such as Postgres databases, Pub/Sub topics, object storage buckets, Redis databases, cron jobs and secrets, as module-level variables in your Python code, and use them through async Python APIs. Encore.py covers infrastructure only. It has no API framework: your HTTP routes stay in the web framework you already use, such as FastAPI, Django or Flask. Encore reads the infrastructure declarations from your code, runs matching infrastructure locally with `encore run`, and gives each test run its own isolated copy with `encore test`. With [the Encore platform](https://encore.dev/docs/platform/introduction), it provisions the same resources in your own AWS or GCP account when you deploy. > **Note** > > Encore.py is in early access. APIs may still change between releases, and a few features are recorded by Encore but not enforced at runtime yet. Each page notes where this applies. ## What you get with Encore.py - **Infrastructure declared in Python**: [Postgres databases](https://encore.dev/docs/py/primitives/databases), [Pub/Sub topics and subscriptions](https://encore.dev/docs/py/primitives/pubsub), [object storage buckets](https://encore.dev/docs/py/primitives/object-storage), [Redis databases](https://encore.dev/docs/py/primitives/redis), [cron jobs](https://encore.dev/docs/py/primitives/cron-jobs), [secrets](https://encore.dev/docs/py/primitives/secrets) and [configuration](https://encore.dev/docs/py/develop/config) are module-level variables. Encore reads them statically, so it knows which service uses which resource and what it does with it. - **Your own web framework**: Encore.py works alongside FastAPI, Django, Flask or plain `http.server`. Encore starts your service and tells it which address to listen on; the routing is yours. - **Local infrastructure, automatically**: `encore run` starts a local Postgres, Pub/Sub broker, object storage and Redis for your app, and applies your migrations. There are no Docker Compose files to maintain. - **Isolated tests**: `encore test` runs `pytest` with its own databases, buckets, topics and Redis keys, created on first use and removed afterwards. - **Observability**: [logging](https://encore.dev/docs/py/observability/logging), [metrics](https://encore.dev/docs/py/observability/metrics) and [tracing](https://encore.dev/docs/py/observability/tracing) that show up in Encore's [local development dashboard](https://encore.dev/docs/dev-dash), with OpenTelemetry providers for your own spans and instruments. ## The infrastructure primitives Each primitive is a module-level variable, declared in the service that owns it. Encore's parser discovers the declaration; the SDK connects it to the provisioned resource the first time you use it. **SQL Database** ```python from encore.storage import postgres db = postgres.Database("orders", migrations="./migrations") async def count_orders() -> int: row = await db.query("SELECT count(*) AS n FROM orders").one() return row["n"] ``` **Pub/Sub** ```python from encore import pubsub from pydantic import BaseModel class Order(BaseModel): order_id: int orders = pubsub.Topic("orders") async def process_order(message: pubsub.Message[Order]) -> None: print("Fulfilling order", message.data.order_id) order_processor = pubsub.Subscription[Order]( orders, "process-orders", handler=process_order ) ``` **Object Storage** ```python from encore.storage import objects uploads = objects.Bucket("uploads") async def save_greeting() -> None: await uploads.upload("hello.txt", b"Hello!", content_type="text/plain") ``` **Redis** ```python from encore.storage import redis sessions = redis.Database("sessions") async def remember(user_id: str) -> None: await sessions.string.set(f"user:{user_id}", b"active", ex=60) ``` **Cron Jobs** ```python from encore import cron _ = cron.Job( "cleanup", path="/internal/cleanup", title="Nightly cleanup", every="24h", ) ``` **Secrets** ```python from encore import secrets stripe_key = secrets.Secret[str]("StripeKey") def stripe_headers() -> dict[str, str]: return {"Authorization": f"Bearer {stripe_key.value}"} ``` A cron job has no runtime client: when it is due, Encore sends a `POST` request for `path` to the service that declares the job, and your framework handles it like any other request. Declarations describe what the application needs, not how it's provisioned. Instance sizes, networking and other deployment settings are configured per environment, so the same code runs locally, in preview environments and in production. ## How Encore reads your code Encore reads your declarations without running your code, so they follow a few rules. The parser reports a violation as an error when it parses the app, before anything runs. - Declare resources as module-level variables, never inside a function, class or conditional. - Pass resource names, and the configuration arguments Encore reads (such as `migrations` or a cron job's `path`), as string or other literals. - Don't alias a declaration (`other = db`) or pass it to a function. To hand a database or Redis database to other code, pass a handle from `read()` or `write()`. Encore records the access level at that call. - Call methods directly on the declaration (`db.query(...)`), rather than storing a method (`q = db.query`). - Use resources from code inside a service directory. Encore attributes each use to the service whose directory contains the file, and grants that service access to the resource. The individual pages describe the rules that are specific to each primitive. ## How an app is laid out An Encore app is a directory with an `encore.app` file at its root. Each service is a directory with an `encore.service.json` file. A single-service app can keep both at the root: ``` my-app/ ├── encore.app ├── encore.service.json ├── pyproject.toml ├── uv.lock ├── main.py ├── resources.py └── migrations/ └── 1_create_orders.up.sql ``` The service file names the service and its language, and tells Encore which HTTP paths the service serves: ```json -- encore.service.json -- { "name": "orders", "lang": "python", "api": [ { "type": "http", "prefix": "/", "name": "orders", "expose": true } ] } ``` See [Services](https://encore.dev/docs/py/primitives/services) for every field, and for apps with several services. `encore app new my-app --template python` creates an app like this with a minimal `http.server` service and no dependencies. To adopt an existing directory, run `encore app init` in it. Either way, add the SDK and your framework to the service's dependencies. ## Installing the SDK The SDK is the `encore-dev` package, imported as `encore`. It needs Python 3.9 or later. Add `encore-dev` to your project's dependencies alongside your framework: ```toml -- pyproject.toml -- [project] name = "orders" version = "0.1.0" requires-python = ">=3.9" dependencies = [ "encore-dev", "fastapi", "uvicorn", ] ``` The package is pure Python. Its native runtime ships with the Encore CLI, which points the SDK at it whenever it starts your code: in `encore run`, `encore test` and `encore exec`. Importing `encore` initializes the runtime and reads its configuration from Encore, so run your app through those commands rather than with `python main.py` on its own. Encore runs your code through the package manager it finds in the service's directory, detected from the lockfile: | Lockfile | Run command | Test command | |---|---|---| | `uv.lock` | `uv run main.py` | `uv run pytest` | | `poetry.lock` | `poetry run python main.py` | `poetry run pytest` | | `Pipfile.lock` or `Pipfile` | `pipenv run python main.py` | `pipenv run pytest` | | `pdm.lock` | `pdm run python main.py` | `pdm run pytest` | | none | `python3 main.py` | `python3 -m pytest` | The entry point is `main.py`, or `app.py` if there is no `main.py`. Without a lockfile, the dependencies must already be importable by `python3`. To run something else, set a custom `run` command in [`encore.service.json`](https://encore.dev/docs/py/primitives/services#running-and-testing-a-service). ## Starting Encore in your app Your entry point starts your web server on the address Encore gives it in the `ENCORE_LISTEN_ADDR` environment variable, as `host:port`. Once your resource declarations are imported, call `await encore.start()` from your framework's startup hook. It starts the handlers of the [Pub/Sub subscriptions](https://encore.dev/docs/py/primitives/pubsub) your service declares on the application's event loop, and returns. With FastAPI, that's the `lifespan` handler: ```python -- main.py -- import os from contextlib import asynccontextmanager import encore import uvicorn from fastapi import FastAPI # Importing resources.py runs its declarations, including any subscriptions. from resources import db @asynccontextmanager async def lifespan(app: FastAPI): await encore.start() yield app = FastAPI(lifespan=lifespan) @app.get("/orders/{order_id}") async def get_order(order_id: int) -> dict: row = await db.query("SELECT id, total FROM orders WHERE id = $1", order_id).one() return row or {} if __name__ == "__main__": host, _, port = os.environ.get("ENCORE_LISTEN_ADDR", "0.0.0.0:8000").rpartition(":") uvicorn.run(app, host=host, port=int(port)) ``` `encore.start()` doesn't take over the event loop or install signal handlers. Calling it again on the same loop is safe, and starts subscriptions imported since the last call. A service without subscriptions doesn't need it, but calling it does no harm. The SDK's I/O methods are `async`. From synchronous code, such as a Django view, call them through a bridge such as `asgiref.sync.async_to_sync`: ```python from asgiref.sync import async_to_sync from .resources import uploads def download(request, name): data = async_to_sync(uploads.download)(name) ... ``` Initialize Encore separately in each worker process: forking a process after the SDK has been imported isn't supported. For how Encore stops your app, see [Graceful shutdown](https://encore.dev/docs/py/develop/shutdown). ## Running and testing - `encore run` parses the app, starts the local infrastructure, applies migrations and starts each service. It restarts a service when its code changes, so turn off your framework's own reloader. - `encore test` runs each service's tests with `pytest`, against infrastructure created for the test run. See [Testing](https://encore.dev/docs/py/develop/testing). - `encore exec` runs a one-off command, such as a seed script or `python manage.py migrate`, attached to the same local infrastructure as `encore run`: ```shell encore exec -- uv run python manage.py migrate ``` ## Next steps - [Services](https://encore.dev/docs/py/primitives/services): the `encore.service.json` file, several services in one app, and service URLs. - [Databases](https://encore.dev/docs/py/primitives/databases): Postgres, migrations, queries, transactions and ORMs. - [Pub/Sub](https://encore.dev/docs/py/primitives/pubsub), [Object Storage](https://encore.dev/docs/py/primitives/object-storage), [Redis](https://encore.dev/docs/py/primitives/redis), [Cron Jobs](https://encore.dev/docs/py/primitives/cron-jobs) and [Secrets](https://encore.dev/docs/py/primitives/secrets). - [Configuration](https://encore.dev/docs/py/develop/config) and [Metadata](https://encore.dev/docs/py/develop/metadata): typed config values and information about the running app. - [Testing](https://encore.dev/docs/py/develop/testing) and [Graceful shutdown](https://encore.dev/docs/py/develop/shutdown). - [Logging](https://encore.dev/docs/py/observability/logging), [Metrics](https://encore.dev/docs/py/observability/metrics) and [Tracing](https://encore.dev/docs/py/observability/tracing). - [App Structure](https://encore.dev/docs/app-structure): how to divide an application into services. --- Source: https://encore.dev/docs/py/primitives/services Section: Building (Encore.py) # Services > Directories that Encore runs, tests and deploys as a unit An Encore service is a directory with an `encore.service.json` file in it. Every Python file in that directory and its subdirectories belongs to the service. Encore runs each service as its own process, gives it its own tests, and grants it access to the infrastructure its code uses. Services are declared in this file rather than in Python, so Encore.py has no service class or decorator. The service's code is an ordinary Python project with your own web framework. ## Defining a service A minimal service file names the service, sets its language, and tells Encore which HTTP paths the service serves: ```json -- encore.service.json -- { "name": "orders", "lang": "python", "api": [ { "type": "http", "prefix": "/", "name": "orders", "expose": true } ] } ``` Next to it go the service's code, its dependencies and its lockfile: ``` my-app/ ├── encore.app └── orders/ ├── encore.service.json ├── pyproject.toml ├── uv.lock ├── main.py ├── resources.py └── migrations/ └── 1_create_orders.up.sql ``` A single-service app can also put `encore.service.json` at the app root, next to `encore.app`. The file is parsed as strict JSON: no comments and no trailing commas. ## The service file | Field | Required | Description | |---|---|---| | `name` | Yes | The service's name. Must be non-empty and unique within the app. | | `lang` | Yes | `"python"` for an Encore.py service. Selects the defaults for `run`, `test` and `build`. | | `description` | No | A sentence or two about what the service is for, shown in Encore's tools. | | `api` | No | The HTTP paths (or API specs) the service serves, and whether the API gateway exposes them. See [Serving HTTP](#serving-http). | | `run` | No | How `encore run` starts the service. See [Running and testing a service](#running-and-testing-a-service). | | `test` | No | How `encore test` runs the service's tests. | | `build` | No | How the service is built for deployment. | | `docker` | No | How `encore build docker` packages the service: `base_image` and `entrypoint`. | | `workers` | No | The service's background processes, which serve no requests. See [Workers](https://encore.dev/docs/py/primitives/workers). | ### Serving HTTP Encore.py has no API framework, so a Python service usually declares a single `http` entry: a path prefix the service serves itself, with whatever framework you choose. Encore doesn't need to know the routes below the prefix; it forwards requests at or below it to the service. ```json "api": [ { "type": "http", "prefix": "/", "name": "orders", "expose": true } ] ``` | Field | Description | |---|---| | `prefix` | The path prefix, starting with `/`. | | `name` | The entry's name, a valid identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to the prefix's last segment, so a prefix of `/` needs an explicit name. | | `expose` | Whether the app's API gateway forwards outside requests to the prefix: `true`, `false` (the default), or a list of gateway names. | | `methods` | The uppercase HTTP methods to forward. Omit it to forward all of them. | A service can list several entries, for example a public prefix and an internal one: ```json "api": [ { "type": "http", "prefix": "/orders", "expose": true }, { "type": "http", "prefix": "/internal", "name": "internal_orders" } ] ``` Instead of an `http` prefix, a service can describe what it serves with an OpenAPI document (`{ "type": "openapi", "spec": "openapi.yaml" }`), a GraphQL schema (`{ "type": "graphql", "schema": "schema.graphql" }`) or protobuf definitions (`{ "type": "grpc", "protos": "proto" }`). Paths are relative to the service directory and can't leave it. ### Running and testing a service `run`, `test` and `build` are each an object with a `type`. For a Python service they default to type `"python"`, which has no options: - **`run`** starts the service's entry point, `main.py` (or `app.py`), through the package manager detected from the lockfile in the service directory: `uv run main.py` with a `uv.lock`, `poetry run python main.py` with a `poetry.lock`, and so on. Without a lockfile it runs `python3 main.py`. See [Installing the SDK](https://encore.dev/docs/py#installing-the-sdk) for the full list. - **`test`** runs `pytest` the same way, for example `uv run pytest`. - **`build`** packages the service's source tree together with its dependency manifests and lockfile, and recreates the environment with the same package manager on deployment. Encore starts the command in the service's directory and passes the address to listen on in the `ENCORE_LISTEN_ADDR` environment variable, as `host:port`. To use another command, set the type to `custom` and give the command as a list of arguments: ```json { "name": "orders", "lang": "python", "run": { "type": "custom", "cmd": ["uv", "run", "python", "-m", "orders"] }, "test": { "type": "custom", "cmd": ["uv", "run", "pytest", "-x"] } } ``` A custom `run` can also have a `build` command, run to completion each time before `cmd` starts. The command is run as given, not through a shell. ## Multiple services An app can have any number of services. Each one is a directory with its own `encore.service.json`, and, for Python, its own dependencies and lockfile, since Encore detects the package manager per service directory: ``` my-app/ ├── encore.app ├── orders/ │ ├── encore.service.json │ ├── pyproject.toml │ ├── uv.lock │ ├── main.py │ └── db.py └── billing/ ├── encore.service.json ├── pyproject.toml ├── uv.lock ├── main.py └── invoices.py ``` Services in one app don't have to share a language: a Python service can sit next to Go, TypeScript or Rust services, and use resources they declare. Start with one service and split it when there's a reason to; [App Structure](https://encore.dev/docs/app-structure) discusses where to draw the boundaries. ## Which service owns what Encore decides which service a file belongs to from its location: a file belongs to the service whose directory contains it. This is what Encore uses to grant access to infrastructure. - **A resource is declared by the service whose directory holds the declaration.** For example, a `postgres.Database("orders", migrations="./migrations")` in `orders/db.py` is the `orders` service's database, with migrations in `orders/migrations`. - **Every use is attributed to the service whose directory contains the code using it**, even if another service imports that file. If `billing` imports a helper from `orders/` that queries a database, the query is still recorded for `orders`. - **Uses must be inside a service.** Code outside every service directory, such as a shared library, can declare resources but can't use them directly. Have the service pass it a handle instead, for example `db.read()` or `db.write()` for a [database](https://encore.dev/docs/py/primitives/databases#handles). To use another service's resource, reference it by name in your own service rather than importing the other service's code. For a database that's [`postgres.Database.named("orders")`](https://encore.dev/docs/py/primitives/databases#using-another-services-database); buckets and topics have the same `named` method. ## Service URLs Your services call each other over HTTP with any client you like. To find another service's address, use `base_url` from `encore.services`: ```python -- billing/invoices.py -- import httpx from encore.services import base_url async def fetch_order(order_id: int) -> dict: async with httpx.AsyncClient() as client: response = await client.get(f"{base_url('orders')}/orders/{order_id}") response.raise_for_status() return response.json() ``` `base_url` returns the service's configured base URL with no trailing slash, so the request path goes right after it. It reads the runtime configuration and makes no network request. A service name the process doesn't know raises `ValueError`. Looking up a service's URL makes the calling service depend on it, and Encore reads that dependency from your code. So: - Pass the service name as a string literal: `base_url("orders")`, not `base_url(name)`. - Call `base_url` directly. Don't store it in a variable or pass the function around. - Call it from code inside a service. - Name a service that exists in the app. Encore reports an unknown name when it parses the app. A service looking up its own URL is allowed, and doesn't count as a dependency. --- Source: https://encore.dev/docs/py/primitives/databases Section: Building (Encore.py) # Databases > Declaring, migrating and querying Postgres Encore treats Postgres databases as logical resources. You declare a database in your code, Encore creates it and applies its migrations, and the SDK gives you an async API to query it. ## Creating a database Declare a database with `postgres.Database` as a module-level variable in your service, and point it at a directory of migrations: ```python -- orders/db.py -- from encore.storage import postgres db = postgres.Database("orders", migrations="./migrations") ``` ```sql -- orders/migrations/1_create_orders.up.sql -- CREATE TABLE orders ( id BIGSERIAL PRIMARY KEY, customer TEXT NOT NULL, total_cents BIGINT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); ``` With this in place, `encore run` creates the `orders` database on a local Postgres cluster and applies the migrations before the service starts. Declaring the database doesn't open a connection: the SDK connects the first time you use it. Encore reads the declaration statically, so: - The declaration must be a module-level assignment, not inside a function, class or conditional. - The name is the only positional argument, and must be a string literal. Database names are unique within the app. - `migrations` is keyword-only, and must be a string literal or `None`. It's a slash-separated path to a subdirectory, relative to the file that declares the database, such as `"./migrations"`. Absolute paths and `..` aren't allowed, and the directory must exist. - Leaving out `migrations` declares a database without Encore-managed migrations. Use this when something else owns the schema, such as [Django's migrations](#using-django). ## Migrations Encore applies each migration once, in order, and records the latest one it applied in the database's `schema_migrations` table. Name each migration file `_.up.sql`: - `` is a number. Migrations run in ascending numeric order, so leading zeros are fine (`0001_create_orders.up.sql`). Each version can only be used once. - `` is a short description of the change. - Files that don't match the pattern are ignored. A matching `_.down.sql` file can revert a migration. Encore never runs down migrations by itself; run them with `encore db migrate --database --down`. ### Changing the schema To change the schema, add a new migration with a higher version than every existing one. Don't edit a migration that has already been applied: Encore doesn't run it again. A new file with a lower version than the latest applied one isn't applied either. ```sql -- orders/migrations/2_add_status.up.sql -- ALTER TABLE orders ADD COLUMN status TEXT NOT NULL DEFAULT 'pending'; ``` The next `encore run` applies it, and so does the next deployment. > **Important** > > During a deployment, code from the previous release may still be running against the migrated schema. Keep migrations compatible with that code: add a column before the code uses it, and remove it only after no running code reads it. ### When a migration fails Encore runs each migration file as a single batch, so the statements of a failed migration are rolled back together, unless the file manages its own transactions. The database is then marked as dirty at the failed version, and Encore won't migrate it any further until you record the last good version yourself: ```shell # Show the recorded version and whether it's dirty. encore db version --database orders # Migration 2 failed: record version 1 as the last applied one, without running anything. encore db migrate --database orders --set=1 ``` Then fix the migration and run the app again. If the very first migration failed, `--set=-1` records that none has been applied. `encore db reset --database orders` drops the local database and creates it again from its migrations, discarding its data. ## Querying Use `query` to read rows and `exec` to run statements. Parameters are passed positionally and referenced as `$1`, `$2`, and so on: ```python async def create_order(customer: str, total_cents: int) -> int: row = await db.query( "INSERT INTO orders (customer, total_cents) VALUES ($1, $2) RETURNING id", customer, total_cents, ).one() return row["id"] async def mark_shipped(order_id: int) -> bool: result = await db.exec( "UPDATE orders SET status = 'shipped' WHERE id = $1", order_id ) return result.rows_affected == 1 ``` Rows are dictionaries keyed by column name. `exec` returns an `ExecResult`, whose `rows_affected` is the number of rows the statement affected; any rows it returns are discarded. `query` returns a `Cursor`, which runs no SQL until you consume it. Consume it once, in one of three ways: - `await cursor.one()` returns the single matching row, or `None` if there is none. It raises `EncoreError` if the query returns more than one row. - `await cursor.all()` returns every row in a list. - `async for row in cursor` streams the rows one at a time. ```python async def recent_orders(customer: str) -> list[dict]: return await db.query( "SELECT id, total_cents FROM orders WHERE customer = $1 ORDER BY id DESC LIMIT 10", customer, ).all() async def total_for(customer: str) -> int: total = 0 async for row in db.query( "SELECT total_cents FROM orders WHERE customer = $1", customer ): total += row["total_cents"] return total ``` Consuming a cursor a second time, or mixing the helpers with iteration, raises `EncoreError`. `one()` and `all()` close the cursor for you, and so does iterating to the end. If you stop iterating early, use the cursor as an async context manager so it closes on exit: ```python async def first_large_order(): async with db.query("SELECT * FROM orders ORDER BY total_cents DESC") as rows: async for row in rows: if row["total_cents"] > 10_000: return row return None ``` ### Types Parameters can be `None`, `bool`, `int`, `float`, `str`, `bytes`, `bytearray`, `list`, `tuple` and `dict`. Encore converts each to the type of the column or placeholder it's bound to: - A `str` can be bound to text, `uuid`, `numeric`, `inet`, `cidr` and enum columns, and to dates and times: `timestamptz` takes an RFC 3339 string such as `"2026-01-31T12:00:00Z"`, `date` takes `"2026-01-31"`. - Any value bound to a `json` or `jsonb` column is stored as JSON, including `dict` and `list`. - Other Python types, such as `datetime` or `Decimal`, aren't supported as parameters yet. Convert them to strings first. Rows come back as these Python types: | Postgres type | Python type | |---|---| | `boolean` | `bool` | | `smallint`, `integer`, `bigint` | `int` | | `real`, `double precision` | `float` | | `text`, `varchar`, enums | `str` | | `bytea` | `bytes` | | `json`, `jsonb` | `dict`, `list` or another JSON value | | `numeric` | `str` | | `timestamp`, `timestamptz` | `str`, in RFC 3339 format | | `date`, `time` | `str` | | `uuid`, `inet`, `cidr` | `str` | | arrays | `list` | | `NULL` | `None` | ## Transactions `begin()` starts a transaction. As an async context manager, the transaction commits when the block exits normally, and rolls back if it raises: ```python async def add_orders(customer: str, totals: list[int]) -> int: async with db.begin() as tx: for total in totals: await tx.exec( "INSERT INTO orders (customer, total_cents) VALUES ($1, $2)", customer, total, ) row = await tx.query( "SELECT count(*) AS n FROM orders WHERE customer = $1", customer ).one() return row["n"] ``` You can also await `begin()` and finish the transaction yourself with `commit()` or `rollback()`. The transaction can't be used after that, even if committing failed: ```python tx = await db.begin() await tx.exec("INSERT INTO orders (customer, total_cents) VALUES ($1, $2)", "ada", 1200) await tx.commit() ``` A transaction that is neither committed nor rolled back is rolled back when it's garbage collected. Close any open cursor before you commit or roll back. ## Connections `query`, `exec` and `begin` on the database take a connection from its pool for each call. To run several statements on the same connection, for example while holding a session-level advisory lock, acquire one with `acquire()`. The connection goes back to the pool when the block exits: ```python async def process_order(order_id: int) -> None: async with db.acquire() as conn: await conn.exec("SELECT pg_advisory_lock($1)", order_id) try: await conn.exec( "UPDATE orders SET status = 'processing' WHERE id = $1", order_id ) finally: await conn.exec("SELECT pg_advisory_unlock($1)", order_id) ``` The connection returns to the pool as it is, so undo any session state you change on it. A connection runs one operation at a time. A streaming cursor borrows its connection or transaction until it's exhausted or closed, so close it before you run the next query on the same connection. `conn.begin()` starts a transaction on the connection, which stays acquired after the transaction ends. Both `acquire()` and `begin()` can also be awaited instead of used with `async with`; close an awaited connection yourself with `await conn.close()`. ## Handles Encore grants each service access to a database based on how the service's code uses it. Using the declaration directly (`db.query(...)`, `db.exec(...)`, `db.begin()`, `db.acquire()`) records read-write access for the service. To hand the database to other code, such as a helper function or a framework dependency, take a handle first. Passing or aliasing the declaration itself is an error, because Encore couldn't tell what the other code does with it: | Method | Access recorded for the service | |---|---| | `db.read()` | Read | | `db.write()` | Read and write | | `db.admin()` | Read, write and admin | Each handle is a `DatabaseRef`, with the same `query`, `exec`, `begin` and `acquire` methods as the database. `DatabaseRef`, `Connection` and `Transaction` all implement `postgres.Queryable`, so a helper can accept any of them. The `Database` declaration deliberately doesn't, so a type checker catches passing it by mistake: ```python from encore.storage import postgres from .db import db async def find_order(conn: postgres.Queryable, order_id: int): return await conn.query("SELECT * FROM orders WHERE id = $1", order_id).one() async def show_order(order_id: int): return await find_order(db.read(), order_id) ``` With FastAPI, a dependency can return a handle: ```python from typing import Annotated from encore.storage import postgres from fastapi import Depends from db import db async def sql_read() -> postgres.DatabaseRef: return db.read() SqlReader = Annotated[postgres.DatabaseRef, Depends(sql_read)] ``` > **Note** > > For now, the access level controls the permissions Encore records, not what the handle can do: all handles share the database's connection pool and credentials, and a `read()` handle can still write. `read(primary=True)` is accepted, but reads aren't routed to replicas yet, so it has no effect. ## Using another service's database By default, each service uses its own database. To use a database declared by another service, reference it by name with `Database.named` at module scope, instead of importing the other service's code: ```python -- reports/db.py -- from encore.storage import postgres orders_db = postgres.Database.named("orders") async def completed_orders() -> int: row = await orders_db.query( "SELECT count(*) AS n FROM orders WHERE status = 'shipped'" ).one() return row["n"] ``` `named` takes the database's name as a string literal. The database is configured where it's declared: its migrations stay with the declaring service, which may be written in another language. Encore records how the `reports` service uses the database, exactly as for its own, and grants it access. ## Connecting with other clients and ORMs `db.connection_string` is a `postgresql://` URL for connecting any Postgres client to the database. It points at a proxy that the Encore runtime runs inside your process, so it's only valid within a process that runs the SDK. Reading it records read-write access for the service. ```python import psycopg from db import db async def count_with_psycopg() -> int: async with await psycopg.AsyncConnection.connect(db.connection_string) as conn: cursor = await conn.execute("SELECT count(*) FROM orders") (count,) = await cursor.fetchone() return count ``` Queries made through the connection string bypass the SDK, so they don't show up in Encore's traces. ### Using Django Django's ORM can use an Encore database through Django's standard PostgreSQL backend and psycopg. Declare the database without `migrations`, since Django owns the schema, and configure `DATABASES` from the connection string: ```python -- storage/resources.py -- from encore.storage import postgres # Django manages the schema through its own migrations. db = postgres.Database("django") ``` ```python -- config/settings.py -- from psycopg.conninfo import conninfo_to_dict from storage.resources import db connection = conninfo_to_dict(db.connection_string) DATABASES = { "default": { "ENGINE": "django.db.backends.postgresql", "NAME": connection.pop("dbname"), "USER": connection.pop("user", ""), "PASSWORD": connection.pop("password", ""), "HOST": connection.pop("host", ""), "PORT": connection.pop("port", ""), "OPTIONS": connection, } } ``` Encore creates the database but doesn't run Django's migrations. Run Django's management commands through `encore exec`, so the SDK gets its configuration: ```shell encore exec -- uv run python manage.py makemigrations encore exec -- uv run python manage.py migrate ``` Other ORMs and query builders, such as SQLAlchemy, connect the same way: give them the connection string, and manage their migrations with their own tools. ## Inserting test and seed data There are a few ways to put data in a database during development. **In a migration**, if every environment should have the data, such as the rows of a lookup table. **With a script**, run against your local database with `encore exec`. The script uses the SDK as your service does. Run it from the service's directory: ```python -- orders/seed.py -- import asyncio from db import db async def main() -> None: for customer in ["ada", "grace"]: await db.exec( "INSERT INTO orders (customer, total_cents) VALUES ($1, $2)", customer, 1000, ) asyncio.run(main()) ``` ```shell encore exec -- uv run python seed.py ``` **At startup, only locally**, by checking the environment with [`encore.app_meta()`](https://encore.dev/docs/py/develop/metadata) in your startup hook. Make the inserts idempotent, since they run on every start: ```python import encore from db import db async def seed_local_data() -> None: if encore.app_meta().environment.cloud == "local": await db.exec( "INSERT INTO orders (id, customer, total_cents) VALUES (1, 'ada', 1000) " "ON CONFLICT (id) DO NOTHING" ) ``` Tests get their own copy of each database. It starts from the migrated schema, including rows inserted by migrations, but not data from scripts or your local database. See [Testing](https://encore.dev/docs/py/develop/testing). ## Inspecting databases The Encore CLI connects to your local databases by their Encore name: - `encore db shell --database orders` opens a `psql` shell. - `encore db conn-uri --database orders` prints a connection URI for other tools. - `encore db query --database orders "SELECT count(*) FROM orders"` runs one query and prints the result. - `encore db proxy` runs a local proxy that forwards to all the app's databases. - `encore db list` lists the app's databases. See the [CLI reference](https://encore.dev/docs/cli/cli-reference) for all `encore db` commands. ## Errors Database errors raise `EncoreError`, a subclass of `RuntimeError` that you can import from `encore`: ```python from encore import EncoreError from db import db async def try_create(order_id: int) -> bool: try: await db.exec( "INSERT INTO orders (id, customer, total_cents) VALUES ($1, 'ada', 1000)", order_id, ) except EncoreError: return False return True ``` `EncoreError` doesn't expose Postgres error codes yet, so the message is the only way to tell, say, a unique violation from a syntax error. --- Source: https://encore.dev/docs/py/primitives/object-storage Section: Building (Encore.py) # Object Storage > Simple and scalable storage for files and unstructured data Object Storage is a simple and scalable way to store files and unstructured data. The most common implementation is Amazon S3, and its semantics are supported by every major cloud provider. Encore.py gives you a cloud-agnostic API for object storage. You declare a bucket in code, and Encore provisions it: a local emulator when you run your app with `encore run`, and Amazon S3 or Google Cloud Storage when you deploy. Every operation is traced automatically. ```python -- uploads/storage.py -- from encore.storage import objects profile_pictures = objects.Bucket("profile-pictures") async def save_picture(user_id: str, data: bytes) -> None: await profile_pictures.upload(f"{user_id}.jpeg", data, content_type="image/jpeg") ``` ## Creating a bucket Create a bucket with `objects.Bucket`, assigned to a module-level variable: ```python from encore.storage import objects profile_pictures = objects.Bucket("profile-pictures", versioned=False, public=False) ``` Encore reads the declaration from your source code to provision the bucket, so a few rules apply: - Declare it as a module-level variable, not inside a function, class, or `if` block. - The name must be a string literal. It's a kebab-case name such as `profile-pictures`, unique within the app. - `versioned` and `public` are keyword-only and default to `False`. Write them as `True` or `False` literals. `versioned=True` keeps previous versions of an object when it is overwritten. `public=True` makes the bucket's objects readable by anyone at a [public URL](#public-buckets). Import the variable from other modules to use the bucket there. ## Uploading objects `upload` stores bytes under a key, replacing any existing object, and returns the stored object's metadata as an `ObjectAttrs`: ```python attrs = await profile_pictures.upload( "ada.jpeg", data, content_type="image/jpeg", ) print(attrs.size, attrs.etag) ``` Both options are keyword-only: - `content_type` sets the object's MIME type. - `if_not_exists=True` rejects the upload with `PreconditionFailed` if an object with that key already exists. ### Streaming uploads To upload a large object without holding it all in memory, pass an async iterable of `bytes` to `upload_stream`. It takes the same `content_type` and `if_not_exists` options: ```python from collections.abc import AsyncIterator from encore.storage import objects videos = objects.Bucket("videos") async def read_chunks(path: str) -> AsyncIterator[bytes]: with open(path, "rb") as f: while chunk := f.read(1024 * 1024): yield chunk async def upload_video(path: str) -> None: await videos.upload_stream("intro.mp4", read_chunks(path), content_type="video/mp4") ``` Chunks are consumed in order, with backpressure: the iterable is only read as fast as the provider accepts data. If the iterable raises or the upload is cancelled, the upload is aborted. Cancellation can't undo an upload the provider has already committed. You own the iterable, so close it yourself if it needs cleanup. ## Downloading objects `download` returns an object's contents as `bytes`: ```python data = await profile_pictures.download("ada.jpeg") ``` To read part of an object, pass `offset` (the zero-based start, default `0`) and `length` (the maximum number of bytes, default to the end): ```python header = await profile_pictures.download("ada.jpeg", offset=0, length=512) ``` ### Streaming downloads `download_stream` yields the object as chunks of `bytes` without collecting it in memory. It takes the same `version`, `offset` and `length` options as `download`. Chunk sizes depend on the provider. If your loop may stop before the stream is exhausted, wrap it in `contextlib.aclosing` so the response is released promptly. For example, to copy a large object: ```python from contextlib import aclosing async def copy_video(source: str, destination: str) -> None: async with aclosing(videos.download_stream(source)) as chunks: await videos.upload_stream(destination, chunks, content_type="video/mp4") ``` ## Listing objects `list` returns an async iterator of `ListEntry` objects, each with a `name`, `size` and `etag`. Filter by key prefix with `prefix`, and cap the number of entries with `limit`: ```python async for entry in profile_pictures.list(prefix="ada", limit=100): print(entry.name, entry.size) ``` If you stop iterating early, close the iterator with `contextlib.aclosing` (or call `aclose()`) to release it promptly: ```python from contextlib import aclosing async def first_large_picture() -> str | None: async with aclosing(profile_pictures.list()) as entries: async for entry in entries: if entry.size > 1_000_000: return entry.name return None ``` ## Object attributes `attrs` returns an object's metadata: its `name`, `version`, `size`, `content_type` and `etag`. It raises `ObjectNotFound` if the object doesn't exist. ```python attrs = await profile_pictures.attrs("ada.jpeg") ``` For convenience, `exists` returns whether an object exists. It returns `False` for a missing object and raises on any other storage failure. ```python if await profile_pictures.exists("ada.jpeg"): ... ``` ## Removing objects ```python await profile_pictures.remove("ada.jpeg") ``` ## Versioned buckets In a bucket declared with `versioned=True`, uploading over an existing object keeps its previous versions. The `version` of the stored object is in the `ObjectAttrs` that `upload` and `attrs` return. `download`, `download_stream`, `attrs`, `exists` and `remove` take a keyword-only `version` to work with a specific version instead of the latest one: ```python documents = objects.Bucket("documents", versioned=True) async def restore(name: str, version: str) -> None: old = await documents.download(name, version=version) await documents.upload(name, old) ``` ## Public buckets Declare a bucket with `public=True` to make its objects publicly readable: ```python avatars = objects.Bucket("public-avatars", public=True) ``` `public_url` returns the URL where an object can be read without authentication. It's synchronous and makes no request: ```python url = avatars.public_url("ada.jpeg") ``` Calling `public_url` on a bucket that isn't declared public is reported as an error when Encore parses your app. When you deploy with Encore Cloud, public buckets are configured for public access and served through a CDN. When self-hosting, see how to configure public buckets in the [infrastructure configuration docs](https://encore.dev/docs/self-host/configure-infra). ## Signed URLs A signed URL grants temporary access to a single object to anyone who has the URL, without any other authentication. It lets clients upload or download content directly to and from the bucket, so large or long-running transfers don't go through your service. `signed_upload_url` returns a URL the client can `PUT` the object's contents to: ```python url = await profile_pictures.signed_upload_url("ada.jpeg", ttl=7200) ``` ```bash curl -X PUT --data-binary @ada.jpeg "" ``` `signed_download_url` returns a URL the client can `GET` the object from, even when the bucket is private: ```python url = await documents.signed_download_url("letter-1234.pdf", ttl=7200) ``` `ttl` is the URL's lifetime in seconds. It defaults to `3600` (one hour) and must be between `1` and `604800` (seven days); anything else raises `InvalidArgument`. For objects in a public bucket, prefer `public_url`: public URLs are served through a CDN, which is usually faster and cheaper. ## Using buckets in your code Encore reads which operations each service performs on a bucket (upload, download, list, and so on) from the method calls in your code. It uses this to draw architecture diagrams and to grant each service exactly the cloud permissions it needs. For this to work, call the methods on the bucket's module-level variable, directly or through an import: ```python from uploads.storage import profile_pictures async def handle_upload(user_id: str, data: bytes) -> None: await profile_pictures.upload(f"{user_id}.jpeg", data) ``` Don't assign the bucket to another variable, return it, or pass it to a function: Encore can't follow it there, and reports the use as an error. ### Buckets declared elsewhere To use a bucket declared by another service, possibly one written in another language, reference it by name with `Bucket.named`: ```python from encore.storage import objects profile_pictures = objects.Bucket.named("profile-pictures") ``` This doesn't create a bucket. Like a declaration, it must be the value of a module-level variable and the name must be a string literal. Encore reports an error if no bucket with that name is declared in the app. ## Error handling Storage failures raise `objects.ObjectsError`, a subclass of `encore.EncoreError`. More specific errors subclass it: - `objects.ObjectNotFound`: the object (or the requested version) doesn't exist. - `objects.PreconditionFailed`: an upload with `if_not_exists=True` found an existing object. - `objects.InvalidArgument`: an argument is out of range, such as a negative `offset` or `limit`, a `ttl` outside its bounds, or a range that starts beyond the end of the object. ```python from encore.storage import objects async def load_picture(user_id: str) -> bytes | None: try: return await profile_pictures.download(f"{user_id}.jpeg") except objects.ObjectNotFound: return None ``` ## Local development `encore run` stores your buckets' objects in a local emulator, so you can develop without any cloud account. Signed URLs and public URLs point at the emulator. The emulator enforces a signed URL's expiry, but doesn't verify its signature. Inspect and manage the local buckets with the `encore objects` commands, such as `encore objects list-buckets`, `encore objects list `, `encore objects cat `, `encore objects upload` and `encore objects rm`. See the [CLI reference](https://encore.dev/docs/cli/cli-reference). Under `encore test`, each test gets its own isolated instance of each bucket it uses. See [Testing](https://encore.dev/docs/py/develop/testing). --- Source: https://encore.dev/docs/py/primitives/pubsub Section: Building (Encore.py) # Pub/Sub > Decoupling services and building asynchronous systems Publishers and subscribers (Pub/Sub) let services communicate by broadcasting events asynchronously. It's a good way to decouple services for better reliability and responsiveness. With Encore.py you declare topics and subscriptions in code, and Encore provisions the infrastructure: a local broker when you run your app with `encore run`, and a cloud service such as GCP Pub/Sub or AWS SNS/SQS when you deploy. Publishing and message handling are traced automatically. ```python -- orders/events.py -- from encore import pubsub from pydantic import BaseModel class OrderPlaced(BaseModel): order_id: int amount: int orders = pubsub.Topic[OrderPlaced]("order-placed") async def send_receipt(message: pubsub.Message[OrderPlaced]) -> None: print("Sending receipt for order", message.data.order_id) receipts = pubsub.Subscription[OrderPlaced](orders, "send-receipt", handler=send_receipt) ``` ```python await orders.publish(OrderPlaced(order_id=1, amount=100)) ``` ## Creating a topic A **topic** is a named channel you publish events to. Create one with `pubsub.Topic`, assigned to a module-level variable: ```python from encore import pubsub signups = pubsub.Topic("user.signed-up") ``` Encore reads the declaration from your source code to provision the topic, so: - Declare it as a module-level variable, not inside a function, class, or `if` block. - The name must be a string literal: lowercase letters, digits, hyphens and periods, unique within the app. - Options must be literals, such as `exactly_once=True`. A topic can be published to from any service and subscribed to from any service. The type argument, as in `pubsub.Topic[OrderPlaced]`, is optional. It lets your type checker check what you publish; it doesn't change how messages are encoded. ### At-least-once delivery By default a topic delivers each event _at least once_ to each subscription: if Encore believes a message wasn't processed, it delivers it again. **Subscription handlers should therefore be [idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning)**, so that handling a message twice has the same effect as handling it once. For example, record in a database which events you've already acted on. ### Exactly-once delivery Set `exactly_once=True` to have the cloud provider minimize redelivery: ```python payments = pubsub.Topic("payment-received", exactly_once=True) ``` Messages can still be redelivered in rare cases, for example when a network failure loses the acknowledgement of a processed message, so handlers that must be correct under all circumstances should still be idempotent. Exactly-once delivery also limits throughput: 300 messages per second per topic on AWS, and at least 3,000 messages per second across the region's topics on GCP. > **Important** > > Exactly-once delivery doesn't deduplicate on the publishing side. If you publish the same message twice, it's delivered twice. ### Ordered topics Topics are unordered by default: messages can be delivered in any order, which lets them be processed in parallel. When messages about the same entity must be handled in the order they were published, declare an `OrderedTopic` and give each message an ordering key when you publish it: ```python cart_events = pubsub.OrderedTopic("cart-events") async def checkout(cart_id: int) -> None: # Delivered in this order, since they share an ordering key. await cart_events.publish({"event": "checkout_started"}, ordering_key=str(cart_id)) await cart_events.publish({"event": "checkout_completed"}, ordering_key=str(cart_id)) ``` Messages with the same ordering key are delivered to each subscription in the order they were published. Messages with different keys are delivered in an unspecified order relative to each other. A message isn't delivered until the one before it with the same key has been processed or dead-lettered, so a failing message holds up the messages behind it ([head-of-line blocking](https://en.wikipedia.org/wiki/Head-of-line_blocking)). Choose a suitable retry policy, and log and alert on failures. Ordered topics are also limited in throughput: 300 messages per second per topic on AWS, and 1 MBps per ordering key on GCP. `OrderedTopic` takes the same `exactly_once` option as `Topic`. The local broker delivers messages in ordering-key order too, so you can test the behavior with `encore run`. ## Publishing events Call `publish` on the topic. It returns the message's ID: ```python message_id = await signups.publish({"user_id": 42}) ``` The message can be: - A top-level [Pydantic](https://docs.pydantic.dev) v2 model, serialized with `model_dump_json()`. - Any JSON-serializable value (dicts, lists, strings, numbers, booleans, `None`), serialized with `json.dumps`. - `bytes`, sent unchanged. Anything else, such as a dataclass or a Pydantic model nested in a plain dict, raises `TypeError`. Wrap nested models in a top-level model instead. You can send string attributes alongside the payload. Subscribers receive them in `message.attributes`: ```python await signups.publish({"user_id": 42}, attributes={"source": "web"}) ``` For an `OrderedTopic`, `ordering_key` is required: `publish(message, ordering_key, attributes=None)`. ### Topics declared elsewhere To publish to or subscribe to a topic declared by another service, possibly one written in another language, reference it by name with `named`: ```python from encore import pubsub signups = pubsub.Topic.named("user.signed-up") ``` Use `pubsub.OrderedTopic.named` for an ordered topic. This doesn't create a topic. Like a declaration, it must be the value of a module-level variable, the name must be a string literal, and Encore reports an error if no topic with that name is declared in the app. Encore reads which services publish to each topic from the `publish` calls in your code, and grants each service the permissions it needs. Call `publish` on the topic's module-level variable, directly or through an import, rather than passing the topic around. ## Subscribing to events A **subscription** receives every event published to its topic and calls your handler for each one. Create one with `pubsub.Subscription`, assigned to a module-level variable: ```python from encore import pubsub from pydantic import BaseModel class SignupEvent(BaseModel): user_id: int signups = pubsub.Topic[SignupEvent]("user.signed-up") async def send_welcome_email(message: pubsub.Message[SignupEvent]) -> None: print("Welcoming user", message.data.user_id) welcome_emails = pubsub.Subscription[SignupEvent]( signups, "send-welcome-email", handler=send_welcome_email, ) ``` The arguments are: - The topic: the variable of a topic declared in, imported into, or referenced with `named` in this module. - The subscription's name, a string literal that is unique among the topic's subscriptions. Use kebab-case, such as `send-welcome-email`. - `handler`, a keyword-only async function that takes a `pubsub.Message`. Pass the function itself (`handler=send_welcome_email`), not the result of calling it. The service whose code declares the subscription is the one that receives its messages. A topic can have many subscriptions, in the same or different services, and each receives every message. ### Starting subscriptions Subscriptions start receiving messages when you call `await encore.start()` on your application's event loop. Call it once at startup, after the modules that declare your subscriptions have been imported. With FastAPI, use the lifespan hook: ```python -- main.py -- from contextlib import asynccontextmanager import encore from fastapi import FastAPI import orders.events # noqa: F401 (declares the subscriptions) @asynccontextmanager async def lifespan(app: FastAPI): await encore.start() yield app = FastAPI(lifespan=lifespan) ``` `encore.start()` returns once the subscriptions are running; handlers run on the same event loop as your application. Calling it again is safe, and starts any subscriptions declared in modules imported since. If a subscription declared in your service has no handler running, for example because its module was never imported, Encore logs a warning shortly after startup. When the service shuts down, Encore stops taking new messages and waits for running handlers to finish. See [Graceful shutdown](https://encore.dev/docs/py/develop/shutdown). ### Message payloads The subscription's type argument decides how each message is decoded before your handler is called: | Declaration | `message.data` | | --- | --- | | `Subscription[MyModel]` | An instance of the Pydantic v2 model, validated with `model_validate_json`. | | `Subscription[pubsub.JSON]` or plain `Subscription` | Ordinary Python values from `json.loads`, without validation. | | `Subscription[bytes]` | The raw payload, unchanged. | The handler's annotation is only for your type checker; it doesn't select the decoder, so keep it in step with the subscription's type argument. Use a Pydantic model when fields need validation or nested models need to be constructed. Dataclasses, unions of models, models nested inside plain JSON containers and string forward references aren't supported; use a top-level model, or `Subscription[bytes]` and decode the payload yourself. A payload that fails to decode raises in place of calling your handler, and is retried like any other failure. ### The message The handler receives a `pubsub.Message` with these fields: | Field | Description | | --- | --- | | `data` | The decoded payload. | | `raw_data` | The original payload bytes. | | `attributes` | The attributes published with the message, as `dict[str, str]`. | | `id` | The message ID, as returned by `publish`. | | `delivery_attempt` | Which delivery attempt this is, starting at 1. | | `publish_time` | When the message was published, as a UTC `datetime`. | | `ordering_key` | The ordering key, for a message published to an `OrderedTopic`; otherwise `None`. | ### Subscription options `Subscription` takes keyword-only options for message retention, retries and concurrency: ```python welcome_emails = pubsub.Subscription[SignupEvent]( signups, "send-welcome-email", handler=send_welcome_email, ack_deadline="60s", min_backoff="5s", max_backoff="5m", max_retries=10, max_concurrency=20, ) ``` | Option | Default | Description | | --- | --- | --- | | `ack_deadline` | `"30s"` | How long a handler has to process a message before it may be redelivered. | | `message_retention` | `"7d"` | How long undelivered messages are kept. | | `min_backoff` | `"10s"` | The shortest delay before retrying a failed message. | | `max_backoff` | `"10m"` | The longest delay before retrying a failed message. | | `max_retries` | `100` | How many times a failed message is retried before it's dead-lettered. `-1` retries forever; `-2` dead-letters a failed message without retrying. | | `max_concurrency` | provider default | The most messages handled at once by each instance of the service. | Durations are strings of numbers and units (`ms`, `s`, `m`, `h`, `d`), such as `"30s"`, `"1h30m"` or `"7d"`. > **Note** > > Encore reads these options from your source code to provision the subscription, so write them as literals in the `Subscription` call. A value computed at run time, such as a variable or a function call, isn't seen, and the default is used instead. ## Error handling If a handler raises an exception, the message is retried after a backoff between `min_backoff` and `max_backoff`. Once it has been retried `max_retries` times, it's moved to the subscription's dead-letter queue, so the subscription can carry on with other messages until you've fixed the problem. You can then release dead-lettered messages to be processed again. Locally, use the `encore pubsub` commands to see and manage what happened to each message: ```shell $ encore pubsub ls # topics, subscriptions and their queues $ encore pubsub queue user.signed-up send-welcome-email --state dead $ encore pubsub show # a message and its delivery attempts $ encore pubsub release user.signed-up send-welcome-email # deliver dead messages again $ encore pubsub publish user.signed-up '{"user_id": 42}' ``` `encore pubsub` also has `history`, `pause`, `resume`, `discard` and `purge`. See the [CLI reference](https://encore.dev/docs/cli/cli-reference). ## Local development and testing `encore run` runs a local Pub/Sub broker, so topics and subscriptions work without any cloud account. Exactly-once delivery isn't emulated locally. Under `encore test`, each test gets its own isolated instance of the topics and subscriptions it uses, so messages published in one test don't reach another. See [Testing](https://encore.dev/docs/py/develop/testing). ## Ensuring consistency between services Writing to your database and publishing an event aren't one transaction: if one succeeds and the other fails, your services can disagree. If that matters, consider the [transactional outbox pattern](https://microservices.io/patterns/data/transactional-outbox.html): write the event to an outbox table in the same transaction as your other changes, and publish it from there. --- Source: https://encore.dev/docs/py/primitives/cron-jobs Section: Building (Encore.py) # Cron Jobs > Run recurring and scheduled tasks When you need to run periodic and recurring tasks, Encore.py lets you declare cron jobs in code. Encore takes care of scheduling them where your app is deployed, and you can monitor their runs in the Encore Cloud dashboard. A cron job is a scheduled HTTP request: when the job is due, Encore sends a `POST` request for the job's path to the service that declares it. Your web framework handles that request like any other. ```python -- users/main.py -- from encore import cron from fastapi import FastAPI app = FastAPI() # Send a welcome email to everyone who signed up in the last two hours. _ = cron.Job( "welcome-email", path="/internal/welcome-emails", title="Send welcome emails", every="2h", ) @app.post("/internal/welcome-emails") async def send_welcome_emails() -> None: # Email everyone who signed up recently. Idempotent: each person # gets at most one welcome email. ... ``` > **Note** > > Cron jobs don't run when you develop locally with `encore run`, or under `encore test`. To try one out, request its path yourself: see [Running a job manually](#running-a-job-manually). ## Declaring a cron job Create a job with `cron.Job`, assigned to a module-level variable. Since you won't use the variable, `_` is a good name for it. ```python from encore import cron _ = cron.Job("accounting-sync", path="/internal/accounting-sync", schedule="0 4 15 * *") ``` The arguments are: - The job's name: a unique kebab-case ID such as `welcome-email`. If you later move the declaration to another file, Encore uses the name to know it's the same job. - `path`: the path to request, starting with `/`. - `title`: an optional human-readable title, shown in the dashboard in place of the name. - Exactly one of `every` and `schedule`, which say when the job runs. See [Schedules](#schedules). Encore reads the declaration from your source code to schedule the job, so: - Declare it as a module-level variable, not inside a function, class, or `if` block, and not as a bare expression without an assignment. - Write the name and every option as string literals. - Declare it inside a service, since the path is requested of the service that declares it. `cron.Job` also raises `ValueError` at import time if both or neither of `every` and `schedule` are set, or if `path` doesn't start with `/`. ## Handling the request When the job is due, Encore sends a `POST` request for `path` to the declaring service. Serve that path with your web framework, as with the FastAPI route above. A Django or Flask view works the same way, as long as it accepts `POST`. Keep in mind: - The path is an ordinary route of your service, reachable by whatever can reach the service. If it's exposed publicly, consider whether anyone calling it could cause harm. - Make the handler idempotent: in rare cases, such as network failures, a job may be run more than once. - The path must be written out in both places, since the job's `path` has to be a string literal. ## Schedules ### Running at an interval `every` runs the job at a fixed interval, as a duration string such as `"10m"`, `"6h"` or `"1h30m"` (units `s`, `m`, `h` and `d`). The runs are laid out over each day starting at midnight UTC, the same every day: `every="6h"` runs at 00:00, 06:00, 12:00 and 18:00 UTC. So that the delay between runs is always the same, the interval must be a whole number of minutes that **divides 24 hours evenly**, and at most 24 hours. `"10m"`, `"45m"` and `"8h"` are fine; `"7h"` and `"50m"` aren't. Encore reports an invalid interval when it parses your app, suggesting the nearest one that works. ### Cron expressions For schedules an interval can't express, such as a specific day of the month or week, use `schedule` with a [cron expression](https://en.wikipedia.org/wiki/Cron) instead of `every`. All times are in UTC. ```python # Run the monthly accounting sync at 04:00 UTC on the 15th day of each month. _ = cron.Job( "accounting-sync", path="/internal/accounting-sync", title="Monthly accounting sync", schedule="0 4 15 * *", ) ``` An expression has the standard five fields: minute (0–59), hour (0–23), day of month (1–31), month (1–12) and day of week (0–6, with Sunday as 0). Each field is `*`, a value, a range such as `9-17`, or a comma-separated list of these, each optionally with a step such as `*/15`. Months and weekdays can also be written by name, such as `JAN` or `mon-fri`. Shorthands such as `@daily` aren't supported. ## Running a job manually Nothing schedules cron jobs locally, so to exercise one, send the request Encore would send: a `POST` for the job's path to the service, while `encore run` is running. For example, with curl against the service's local URL: ```shell $ curl -X POST http://localhost:/internal/welcome-emails ``` If you use an AI agent connected to Encore's MCP server, it can run a job by name with the `cron_run` tool, which sends the same request through Encore's local service proxy. In tests, call the function that serves the path, or request the path with your framework's test client. --- Source: https://encore.dev/docs/py/primitives/workers Section: Building (Encore.py) # Workers > Background processes of a service A worker is a long-running process of a service that serves no requests. It runs next to the service's main process, from an entrypoint of its own in the service's code, and does whatever that code does: consume Pub/Sub messages, run a Temporal worker, poll a queue, reconcile state on a timer. A worker *is* its service at runtime. It has the service's secrets, configs, databases and permissions, and it calls other services as the service does. There is nothing to grant or share: code a worker uses is the service's code. ## Enabling workers Workers are experimental: they work, but how they are declared may still change. An app uses them only by enabling the `workers` feature in its `encore.app.json`: ```json -- encore.app.json -- { "features": ["workers"] } ``` Without it, Encore refuses to parse an app that declares workers, with an error saying what to add. ## Declaring workers Workers are declared in the service's `encore.service.json`, by name: ```json -- orders/encore.service.json -- { "name": "orders", "lang": "python", "api": [ { "type": "http", "prefix": "/", "name": "orders", "expose": true } ], "workers": { "fulfillment": { "entry": "fulfillment_worker.py" }, "temporal": { "entry": "-m orders.temporal_worker", "shutdown_timeout": "10m" }, "reconciler": { "entry": "reconciler.py", "instances": "one" } } } ``` | Field | Required | Description | |---|---|---| | `entry` | Yes, unless `run` is given | The worker's entrypoint: a file relative to the service directory, or `-m module` to run a module. | | `description` | No | A sentence or two about what the worker is for, shown in Encore's tools. | | `instances` | No | `"any"` (the default), or `"one"` for code that must never run twice at once, such as a leader loop. A deployment then runs a single copy of it, and stops the old one before starting a new one. This is best effort: take a lock where correctness depends on it. | | `shutdown_timeout` | No | How long the worker may take to stop once told to, as `"30s"` or `"10m"`. Defaults to the deployment's graceful shutdown. | | `run` | No | How `encore run` starts it, as a service's `run`, instead of from `entry`. | | `docker` | No | `{ "entrypoint": [...] }`: the command its process runs in a Docker image, instead of one derived from `entry`. | A worker's name is lowercase letters, digits and underscores, unique within its service. Within the app, Encore calls it `.`, as in `orders.fulfillment`. ## Writing a worker A worker's entrypoint is an ordinary Python program that starts Encore and runs until it is told to stop. One that only consumes Pub/Sub messages needs nothing more than `encore.run()` without an app: ```python -- orders/fulfillment_worker.py -- import encore import subscriptions # declares the subscriptions this worker consumes encore.run() ``` One with work of its own runs it until shutdown begins, then lets it finish. See [Graceful shutdown](https://encore.dev/docs/py/develop/shutdown) for how: ```python -- orders/reconciler.py -- import asyncio import signal import encore from encore import shutdown async def reconcile(stop: asyncio.Event) -> None: while not stop.is_set(): ... # compare and fix state, using Encore resources try: await asyncio.wait_for(stop.wait(), timeout=60) except asyncio.TimeoutError: pass async def main() -> int: await encore.start() stop = asyncio.Event() asyncio.get_running_loop().add_signal_handler(signal.SIGTERM, stop.set) task = asyncio.create_task(reconcile(stop)) await stop.wait() shutdown.begin(cooperative=True) await task shutdown.mark_application_drained() return (await shutdown.completed()).code() if __name__ == "__main__": raise SystemExit(asyncio.run(main())) ``` `encore.env.app_meta().deploy.worker` is the name of the worker a process runs, and `None` in the service's main process. ## Consuming Pub/Sub messages A subscription names the worker that consumes it with `worker`: ```python -- orders/subscriptions.py -- from encore import pubsub from topics import orders, Order async def fulfil(message: pubsub.Message[Order]) -> None: ... fulfil_orders = pubsub.Subscription[Order]( orders, "fulfil-orders", handler=fulfil, worker="fulfillment" ) ``` Only that worker's process takes the subscription's messages. The service's main process, and its other workers, take none, even when their code imports the module that declares it. Without `worker`, the service's main process consumes the subscription, as it always has. Encore reads `worker` from the source, so write it as a string, and it must name a worker the service declares. ## A service with only workers A service that serves nothing has no main process: set its `run` to `"none"`. ```json -- indexer/encore.service.json -- { "name": "indexer", "lang": "python", "run": "none", "workers": { "index": { "entry": "index_worker.py" } } } ``` Such a service can't declare an `api`, a frontend or a gateway, or cron jobs, which call a path of their service. Each subscription it declares needs a `worker`, since no main process would consume it. ## Cron jobs A cron job calls an HTTP path of its service, which only the main process serves, so it can't target a worker. Declare it in the service, and have its endpoint publish a message a worker consumes if the work belongs in the worker. ## Running workers locally Under `encore run`, a service's workers start when the service does, and keep running while anything keeps the service up. Unlike the main process, a worker is never stopped for being idle, since no request would start it again. A message waiting on a subscription a worker consumes starts the worker too. Each worker shows up in `encore run` and the dashboard as `.`. Its log records are its service's, with an `encore.worker.name` attribute naming the worker. It counts as running once it has stayed up for 250 milliseconds; one that exits sooner failed to start, and its exit and last lines are shown. A worker that crashes stays down until the next rebuild, as a service does. When the service's source changes, its workers are rebuilt with it: each is stopped, then started again, so a worker never runs twice. ## Deploying workers `encore build docker` packages a service's workers in its image, and the image runs them beside the service. A deployment can also run them on their own, in containers that run only workers. Its infra config's `hosted_workers` lists the workers a container runs, by `.`, and `encore infra config generate` lists each worker where its service runs. A worker that runs with its service scales with it. On compute that throttles CPU between requests, as Cloud Run does by default, a worker sharing a container with its service needs CPU allocated all the time. --- Source: https://encore.dev/docs/py/primitives/redis Section: Building (Encore.py) # Redis > Fast in-memory data structures for caching, counters, sessions and more [Redis](https://redis.io) is an in-memory data store, commonly used for caching, rate limiting, session storage, leaderboards and queues. With Encore.py you declare a Redis database in code, and Encore provisions it: a local Redis emulator when you run your app with `encore run`, and a managed Redis when you deploy. Every command is traced automatically. ```python -- sessions/store.py -- from encore.storage import redis sessions = redis.Database("sessions") async def save_session(session_id: str, user_id: str) -> None: await sessions.string.set(f"session:{session_id}", user_id, ex=3600) async def load_session(session_id: str) -> str | None: return await sessions.string.get[str](f"session:{session_id}") ``` > **Note** > > **Coming from Encore.go or Encore.ts?** Their caching API ([Go](https://encore.dev/docs/go/primitives/caching), [TypeScript](https://encore.dev/docs/ts/primitives/caching)) has cache clusters and typed keyspaces. Encore.py has Redis databases instead: you declare a `redis.Database` and call Redis commands on it directly, choosing your own keys and how values are encoded. There are no keyspaces, and no eviction policy option yet. ## Creating a database Create a database with `redis.Database`, assigned to a module-level variable: ```python from encore.storage import redis sessions = redis.Database("sessions") ``` Encore reads the declaration from your source code to provision the database, so: - Declare it as a module-level variable, not inside a function, class, or `if` block. - The name must be a string literal. It's a kebab-case name such as `sessions`, unique within the app. `Database` takes no other options yet: persistence and eviction policies can't be configured. Import the variable from other modules to use the database there. ## Running commands Commands are grouped into families, which are properties of the database. Call a command on its family and await it: ```python await sessions.string.set("user:1", "Ada", ex=60) name = await sessions.string.get("user:1") # b"Ada" await sessions.hash.hset("user:1:profile", ("name", "Ada"), ("lang", "en")) await sessions.list.rpush("queue", "job-1", "job-2") await sessions.sorted_set.zadd("leaderboard", (100, "ada"), (85, "grace")) await sessions.generic.expire("queue", 300) ``` | Family | Commands for | | --- | --- | | `generic` | Keys of any type: `exists`, `delete`, `expire`, `ttl`, `rename`, `scan`, … | | `string` | Strings and counters: `get`, `set`, `mget`, `incr`, `incrby`, `append`, … | | `bitmap` | Bits of strings: `getbit`, `setbit`, `bitcount`, `bitpos`, `bitop` | | `hash` | Hashes: `hget`, `hset`, `hgetall`, `hincrby`, `hdel`, … | | `list` | Lists: `lpush`, `rpush`, `lpop`, `rpop`, `lrange`, `lmove`, … | | `set` | Sets: `sadd`, `srem`, `smembers`, `sismember`, `sinter`, … | | `sorted_set` | Sorted sets: `zadd`, `zrange`, `zrangebyscore`, `zincrby`, `zrank`, … | | `geo` | Geospatial indexes: `geoadd`, `geodist`, `geosearch`, … | | `stream` | Streams: `xadd`, `xrange`, `xread`, `xreadgroup`, `xack`, … | | `json` | RedisJSON documents: `set`, `get`, `arrappend`, `numincrby`, … | | `pubsub` | Redis channels: `publish`, `subscribe`, `psubscribe` | Methods are named after the Redis commands they run, and their options are keyword arguments, such as `set(key, value, ex=60, nx=True)`. Redis errors raise `encore.EncoreError`. Keys are plain strings. Encore namespaces them behind the scenes, for example to isolate tests from each other, but you always read and write the unprefixed names: `generic.scan` returns your keys as you wrote them. ### Multiple values Commands that take several values accept them as extra positional arguments, with a tuple for each grouped value such as a hash field and its value, or a sorted-set score and its member: ```python await sessions.string.mset(("first", "Ada"), ("last", "Lovelace")) values = await sessions.string.mget("first", "last") await sessions.set.sadd("tags", "python", "redis") ``` To pass an existing collection, unpack it with `*`, as in `sessions.set.sadd("tags", *tags)`. ## Values You can store `bytes`, `str`, `int` and `float` values. Strings are encoded as UTF-8, integers as decimal text, floats using `repr()`, and bytes are stored unchanged. Booleans aren't supported as values. Reads return `bytes` by default. To decode a stored value as another type, put the type in square brackets after the command: ```python await sessions.string.set("visits", 42) visits = await sessions.string.get[int]("visits") # int | None await sessions.list.rpush("names", "Ada", "Grace") names = await sessions.list.lrange[str]("names", 0, -1) # list[str] top = await sessions.sorted_set.zrange[str]("leaderboard", 0, 9, withscores=True) # list[tuple[str, float]] ``` The type applies to stored values: list elements, hash values, set members, and so on. Keys, hash field names, scores and other metadata keep their own types; for example, `hash.hgetall[int]` returns `dict[bytes, int]`. A missing value is `None`, and an empty collection is empty. A value that can't be decoded as the requested type (invalid UTF-8, or text that isn't a number) raises instead of returning `None`. Commands that change data and also return it, such as `getdel` or `lpop`, run before the result is decoded, so a decoding error doesn't undo them. ### Custom value types To store your own types, give them an `encode` method that returns `bytes` and a `decode` class method that builds an instance from `bytes`. This is the `redis.RedisValue` protocol; you don't need to inherit from anything: ```python import json from dataclasses import dataclass @dataclass class User: name: str def encode(self) -> bytes: return json.dumps({"name": self.name}).encode() @classmethod def decode(cls, data: bytes) -> "User": return cls(**json.loads(data)) await sessions.string.set("user:1", User("Ada")) user = await sessions.string.get[User]("user:1") # User | None ``` Redis doesn't record which type a value was stored as, so use the same type when you read it back. ## Read and write handles Encore reads which Redis commands each service uses from your code, to know which services read and which write each database. That's why a command must be called directly on its family, as in `sessions.string.get(...)`: Encore can see both the family and the command at that call. If you store a family in a variable or pass it to a function, Encore can't tell which commands run through it, and reports an error. To hand access to a database to other code, such as a helper function or a FastAPI dependency, take a handle first: - `sessions.read()` returns a `redis.ReadOnlyDatabaseRef`, which has only the read commands of each family. - `sessions.write()` returns a `redis.DatabaseRef`, which has every command. Handles can be passed around and stored freely; Encore records the access at the `read()` or `write()` call. ```python from typing import Annotated from encore.storage import redis from fastapi import Depends, FastAPI from sessions.store import sessions app = FastAPI() async def cache_reader() -> redis.ReadOnlyDatabaseRef: return sessions.read() async def count_visits(db: redis.DatabaseRef, page: str) -> int: return await db.string.incr(f"visits:{page}") @app.get("/pages/{page}") async def get_page( page: str, reader: Annotated[redis.ReadOnlyDatabaseRef, Depends(cache_reader)] ): cached = await reader.string.get[str](f"page:{page}") visits = await count_visits(sessions.write(), page) return {"cached": cached, "visits": visits} ``` A read handle rejects writes at run time too, not only in your type checker. > **Note** > > `read(primary=True)` asks for reads from the primary server even where replicas exist. Encore currently sends all reads to the primary, so the flag has no effect yet. ## Redis channels The `pubsub` family is Redis's own publish/subscribe: fire-and-forget messages to whoever is subscribed to a channel at that moment. It's separate from Encore's [Pub/Sub](https://encore.dev/docs/py/primitives/pubsub) topics, which store messages durably and retry failed deliveries. Publish with `publish`, which returns how many subscribers received the message: ```python await sessions.pubsub.publish("chat:lobby", "hello") ``` `subscribe` opens a dedicated connection and returns a subscription you iterate asynchronously. Use it as an async context manager so the connection is closed when you're done, even if your code raises or is cancelled: ```python async def listen() -> None: async with await sessions.pubsub.subscribe("chat:lobby") as messages: async for message in messages: print(message.channel, message.payload) # both bytes ``` `psubscribe` subscribes to channels matching glob patterns, such as `chat:*`. Both take several channels or patterns, and a type for the payloads, as in `subscribe[str]("chat:lobby")`. Each message has a `channel` (always `bytes`) and a decoded `payload`. You can also call `receive()`, which returns the next message or `None` once the subscription is closed, and `close()`. ## JSON The `json` family works with [RedisJSON](https://redis.io/docs/latest/develop/data-types/json/) documents. Values are Python JSON values, encoded and decoded for you: ```python await sessions.json.set("profile:1", "$", {"name": "Ada", "active": True}) names = await sessions.json.get("profile:1", "$.name") # ["Ada"] await sessions.json.numincrby("profile:1", "$.logins", 1) ``` A JSONPath (starting with `$`) can select several values, so replies to one are lists of matches. > **Important** > > JSON commands need a Redis server with RedisJSON. The local emulator that `encore run` and `encore test` use doesn't support them yet. ## Connecting other clients `connection_string` returns the database's Redis URL, including credentials, for use with another Redis client: ```python url = sessions.connection_string ``` It's available on the database and on write handles, not on read handles, and grants full read-write access. Clients using it bypass Encore: they don't apply Encore's key namespacing, so they may not see the same keys as Encore's commands (under `encore test`, for example), and their commands aren't traced. ## Not supported yet The Python SDK doesn't yet have: - Persistence or eviction options on `Database`. - Transactions (`MULTI`/`EXEC`), Lua scripts, or pipelining. - Read replica routing (see `read(primary=True)` above). - A way to reference a Redis database by name, as `Bucket.named` and `Topic.named` do. Import the declaring module's variable instead. ## Local development and testing `encore run` runs a local Redis emulator for your databases, so you don't need to install Redis. Under `encore test`, each test gets its own namespace of keys and channels on a shared local server, so tests don't see each other's data. See [Testing](https://encore.dev/docs/py/develop/testing). --- Source: https://encore.dev/docs/py/primitives/secrets Section: Building (Encore.py) # Secrets > Keep API keys and credentials out of your code API keys, database passwords and private keys don't belong in source code. Encore.py lets you declare a secret in code and give it a value outside it: on your machine, or in Encore Cloud for each environment. ## Declaring a secret Declare a secret as a module-level variable, with the type of its value, `str` or `bytes`: ```python -- billing/secrets.py -- from encore.secrets import Secret stripe_key = Secret[str]("StripeKey") ``` Read the value with `.value` where you need it: ```python -- billing/payments.py -- import httpx from billing.secrets import stripe_key async def charge(customer_id: str, amount: int) -> None: async with httpx.AsyncClient() as client: await client.post( "https://api.stripe.com/v1/charges", headers={"Authorization": f"Bearer {stripe_key.value}"}, data={"customer": customer_id, "amount": amount, "currency": "usd"}, ) ``` A few rules apply, because Encore reads secret declarations from the source: - Declare secrets at module level, not inside functions or classes. - The name must be a string literal and an identifier: a letter or underscore, then letters, digits and underscores. - The type argument is required. `Secret[str]` decodes the value as UTF-8 and `Secret[bytes]` returns it unchanged. A variable annotation alone doesn't count: `key: Secret[str] = Secret("Key")` raises `TypeError` when you read `.value`. - Use the secret by reading `.value` (or `.name`). Encore records which services read which secrets from these accesses, so it rejects passing the secret object around as a value. Pass the value instead. Reading `.value` raises `KeyError` if the secret has no value in the current environment. The value is looked up on first read and cached. ## Secret scopes The class you declare a secret with decides who it belongs to: | Class | Belongs to | Where you can declare it | | --- | --- | --- | | `Secret[T]` | The service that declares it. Two services can each have a secret with the same name and different values. | Inside a service | | `SharedSecret[T]` | The whole app. Encore records each service that reads it. | Anywhere | | `GlobalSecret[T]` | The whole app, available to every service. Reads are not recorded. | Anywhere | ```python from encore.secrets import GlobalSecret, Secret, SharedSecret api_key = Secret[str]("ApiKey") # this service's own signing_key = SharedSecret[bytes]("SigningKey") # shared by several services sentry_dsn = GlobalSecret[str]("SentryDSN") # e.g. for library code outside services ``` Service secrets have a namespace of their own, separate from shared and global secrets, so a service's `ApiKey` and a shared `ApiKey` are different secrets, and neither falls back to the other. Shared and global secrets use the same namespace. Declare each shared or global secret once in the app, and import the declaration wherever you need it. ## Setting secret values Use `encore secret set`. It reads the value from stdin, so you can pipe it in, or it prompts for it without echoing when run in a terminal. ### On your machine By default, `encore secret set` writes a value for your machine only: ```shell $ encore secret set SigningKey # a shared or global secret $ encore secret set --service billing ApiKey # the billing service's own secret $ encore secret set billing.ApiKey # the same thing ``` Run `encore secret set` with no name to be asked for each declared secret that has no value on your machine. These values are stored in `.secrets.local.toml` in your app's root, next to `encore.app`. The file is meant to be gitignored. You can also edit it directly: ```toml -- .secrets.local.toml -- [shared] SigningKey = "local-signing-key" [services.billing] ApiKey = "sk_test_123" ``` `[shared]` holds shared and global secrets, and `[services.]` holds each service's own. Values are plain strings. A running service keeps the values it started with. When a value it uses changes, `encore run` reports the service as stale until it restarts. ### In Encore Cloud To store a value in Encore Cloud, name the environment types or environments it's for: ```shell $ encore secret set --type production SigningKey $ encore secret set --type development,preview,local SigningKey $ encore secret set --env staging SigningKey ``` The environment types are `production` (`prod`), `development` (`dev`), `preview` (`pr`) and `local`. A value set for a named environment with `--env` takes precedence over the value for its type. For an app linked to Encore Cloud, values for the `local` type reach every developer's `encore run` and `encore test`; a value in `.secrets.local.toml` overrides them on your machine. You can also manage values in the [Encore Cloud dashboard](https://app.encore.dev), under **Settings > Secrets**. > **Important** > > Encore Cloud doesn't store per-service secret values yet. A service's own `Secret` can only get a value on your machine, with `--service`. For values you need in deployed environments, use `SharedSecret` or `GlobalSecret`. ### Checking and removing values ```shell $ encore secret list # which secrets have values, and where $ encore secret list --missing # only declared secrets without a value; exits non-zero if any $ encore secret unset SigningKey # remove this machine's value $ encore secret unset --type production SigningKey ``` `encore secret list` never prints values. `encore secret unset` takes the same target flags as `set`. ## Secrets in tests `encore test` gives your tests the same values as `encore run`: the `local` values from Encore Cloud, overridden by `.secrets.local.toml`. A secret with no value raises `KeyError` when the test reads it. ## Secrets in configuration A [config](https://encore.dev/docs/py/develop/config) can have fields that read a secret rather than hold a value, so code reading the config gets the secret along with its other settings. See [Secrets in config](https://encore.dev/docs/py/develop/config#secrets-in-config). --- Source: https://encore.dev/docs/py/develop/metadata Section: Building (Encore.py) # Metadata > Information about the app and the environment it's running in Encore tries to give your code the same environment wherever it runs, but sometimes it helps to know more: which environment you're in, which cloud it runs on, or where the app can be reached. The `encore.env` module provides this information. ## Application metadata Call `app_meta()` to get an `AppMeta` object describing the running application. It's also available as `encore.app_meta()`. ```python from encore import env meta = env.app_meta() print(meta.app_id, meta.environment.name, meta.environment.type) ``` The metadata is read on first use, and every later call returns the same object. All the metadata classes are frozen dataclasses. `AppMeta` has these attributes: | Attribute | Description | | --- | --- | | `app_id` | The Encore application ID. Empty if the app isn't linked to Encore Cloud. | | `app_slug` | The application's human-readable slug, as shown in the Encore Cloud dashboard and used by the CLI. Empty if the app isn't linked. Unlike `app_id`, it can change if the app is renamed. Locally the two are the same. | | `gateways` | The API gateways serving the app, as a `dict` of gateway name to `GatewayMeta`. | | `environment` | The [environment](https://encore.dev/docs/platform/deploy/environments) the app is running in, as `EnvironmentMeta`. | | `build` | The build the running code came from, as `BuildMeta`. | | `deploy` | This deployment, as `DeployMeta`. | ### Environment `meta.environment` is an `EnvironmentMeta` with: - `name`: the environment's name. For local development it's `"local"`. - `type`: one of `"production"`, `"development"`, `"ephemeral"` or `"test"`. It's `"development"` under `encore run` and `"test"` under `encore test`. - `cloud`: one of `"aws"`, `"gcp"`, `"azure"`, `"encore"` or `"local"`. It's `"local"` for local development. ### Gateways An app can have several API gateways, so there's no single base URL for its API. Look up the gateway you need by name: ```python gw = env.app_meta().gateways.get("api") if gw is not None: print("api is at", gw.base_url) ``` Each `GatewayMeta` has a `name` and a `base_url`. Locally, `base_url` is `http://localhost:`, typically `http://localhost:4000`. If the environment uses a custom domain, `base_url` uses it, but note that it reflects the domain at the time of deployment. A process that serves no gateway, such as a worker, gets an empty `dict`. To call another service directly rather than through a gateway, use [`encore.services.base_url`](https://encore.dev/docs/py/primitives/services). ### Build and deployment - `build.revision`: the version control revision the app was built from. - `deploy.id`: the deployment's unique ID, generated by Encore Cloud. - `deploy.time`: when this build was deployed to the environment, as a timezone-aware `datetime`. - `deploy.hosted_services`: the services this deployment hosts, as a list of `HostedService`, each with a `name`. ## Example use cases ### Behaving differently by environment Use the environment type rather than its name when behavior should follow the kind of environment, so it keeps working as you add environments: ```python from encore import env def send_email(to: str, subject: str, body: str) -> None: if env.app_meta().environment.type != "production": print(f"email to {to}: {subject}") # don't email real users outside production return # ... send it for real ``` ### Using cloud-specific services Encore supports a common set of infrastructure on every cloud. For services it doesn't cover, branch on the cloud the environment runs on. For example, to write audit logs to a data warehouse: ```python from encore import env async def write_audit_log(entry: dict) -> None: cloud = env.app_meta().environment.cloud if cloud == "gcp": await write_to_bigquery(entry) elif cloud == "aws": await write_to_redshift(entry) else: write_to_file(entry) ``` For values that differ between environments, prefer [configuration](https://encore.dev/docs/py/develop/config) over checking the environment in code. --- Source: https://encore.dev/docs/py/develop/testing Section: Building (Encore.py) # Testing > pytest against real, isolated infrastructure Encore.py apps are tested with [pytest](https://docs.pytest.org), run through `encore test`. Encore gives the test process its own databases, buckets, Pub/Sub topics and Redis keys, created when the tests first use them and removed when the process exits. Your tests use the same declarations and the same SDK calls as your service, against real Postgres and real emulators, so most tests of Encore code are integration tests. ## Running tests Run all tests from anywhere in the app: ```shell encore test ``` For each Python service, `encore test` runs `pytest` in the service's directory, through the package manager detected from its lockfile: `uv run pytest` with a `uv.lock`, `poetry run pytest` with a `poetry.lock`, and so on, or `python3 -m pytest` without one. With several services, it tests them in parallel and prefixes each line of output with the service's name. To change the command, set a custom `test` in [`encore.service.json`](https://encore.dev/docs/py/primitives/services#running-and-testing-a-service). To choose which services to test, pass a service-set expression with `--services`, and limit how many run at once with `--concurrency`: ```shell encore test --services orders encore test --services 'orders | billing' --concurrency 1 ``` To pass arguments to pytest, give the whole command after `encore test`. Without `--services`, the command runs as given, in the current directory; with `--services`, it runs in each selected service's directory: ```shell # From the service's directory: encore test uv run pytest -k create_order -x # In the orders service's directory, wherever you are in the app: encore test --services orders uv run pytest tests/test_orders.py ``` Always run tests through `encore test`. Importing `encore` initializes the SDK's runtime, which gets its configuration from Encore; running `pytest` on its own fails at that import. ## Setting up pytest Add pytest to the service's development dependencies, together with [pytest-asyncio](https://pytest-asyncio.readthedocs.io) for testing async code. Most SDK methods are `async`, so most tests are too: ```toml -- pyproject.toml -- [dependency-groups] dev = ["pytest>=8", "pytest-asyncio>=0.24"] [tool.pytest.ini_options] testpaths = ["tests"] pythonpath = ["."] ``` `pythonpath = ["."]` lets tests in `tests/` import the service's modules, such as `from resources import db`, the same way the service does. Mark async tests with `@pytest.mark.asyncio`, and declare async fixtures with `@pytest_asyncio.fixture`. To skip the markers, set `asyncio_mode = "auto"` in `[tool.pytest.ini_options]`, and pytest-asyncio treats every `async def` test and fixture as asyncio. ## Testing database code Test code queries the database through its declaration, as the service does: ```python -- tests/test_users.py -- from uuid import uuid4 import pytest from resources import db @pytest.mark.asyncio async def test_add_user() -> None: name = f"user-{uuid4().hex}" await db.exec("INSERT INTO users (name) VALUES ($1)", name) row = await db.query("SELECT name FROM users WHERE name = $1", name).one() assert row == {"name": name} ``` Each test process gets its own copy of each database, with every migration applied, including rows that migrations insert. Encore migrates a template database once and clones it for each test process, so starting from a fresh database is fast. The template is reused across runs until the migrations change. ## What Encore sets up Nothing is created until a test uses it. On first use in a test process, Encore creates an instance of the resource for that process: | Resource | Isolation | |---|---| | [Database](https://encore.dev/docs/py/primitives/databases) | A clone of the migrated template database. | | [Bucket](https://encore.dev/docs/py/primitives/object-storage) | A key prefix of its own in the local object storage. | | [Pub/Sub topic and subscription](https://encore.dev/docs/py/primitives/pubsub) | Names of their own in the local broker. | | [Redis database](https://encore.dev/docs/py/primitives/redis) | A namespace of its own on the local Redis server. | When the test process exits, Encore removes everything it created for it. Test databases run on a separate Postgres cluster, configured without durability (`fsync` is off) since tests don't need it. Encore keeps the cluster running for a while after a test run, so the next run starts quickly. During `encore test`, the environment type is `"test"`, which [`encore.app_meta()`](https://encore.dev/docs/py/develop/metadata) reports as `environment.type`. ## Keeping tests independent Isolation is per test process, not per test: every test in one pytest run shares the same database, bucket, topics and Redis namespace. Encore.py doesn't yet have per-test databases or fixtures for saved test data. So write tests that don't depend on each other's data: - Use unique keys, names and IDs in each test, for example with `uuid4()`. - Clean up in a fixture, so the cleanup also runs when a test fails. - Don't assert on totals across a table, such as `count(*)`, that other tests also write to. Filter on the rows the test created. This fixture gives each object storage test a key of its own, and removes the object afterwards: ```python -- tests/test_objects.py -- from contextlib import suppress from uuid import uuid4 import pytest import pytest_asyncio from encore.storage import objects from resources import uploads @pytest_asyncio.fixture async def object_key(): key = f"tests/{uuid4().hex}.txt" yield key with suppress(objects.ObjectNotFound): await uploads.remove(key) @pytest.mark.asyncio async def test_upload_and_download(object_key: str) -> None: attrs = await uploads.upload(object_key, b"abcdef", content_type="text/plain") assert attrs.size == 6 assert await uploads.download(object_key) == b"abcdef" ``` Separate test processes are isolated from each other, so services tested in parallel by `encore test`, or pytest workers in separate processes, don't see each other's data. With [Hypothesis](https://hypothesis.readthedocs.io), a function-scoped fixture runs once per test, not once per generated example. Create the key inside the test instead, for example with an async context manager that cleans up after each example. ## Testing Pub/Sub subscriptions Subscription handlers only consume messages once `encore.start()` has been awaited, and a test process doesn't call it for you. To test a handler's logic, call the handler function directly. ## Testing Django apps Django tests run under `encore test` like any other pytest suite. Configure Django before the first test, for example by importing the WSGI application, and apply Django's migrations to the test database yourself, since Encore only applies Encore-managed migrations: ```python -- tests/test_orm.py -- from uuid import uuid4 import pytest from django.core.management import call_command from django.db import connections from django.test import Client from config.wsgi import application # noqa: F401 (configures Django) from storage.models import Note @pytest.fixture(scope="module") def migrated_database(): # Encore supplies the isolated database; Django owns its schema. call_command("migrate", interactive=False, verbosity=0) yield connections.close_all() @pytest.fixture def note_title(migrated_database): title = f"note-{uuid4().hex}" yield title Note.objects.filter(title=title).delete() def test_create_note(note_title): response = Client().post("/notes/", {"title": note_title}) assert response.status_code == 201 assert Note.objects.get(pk=response.json()["id"]).title == note_title ``` The database Encore provides is already isolated, so this uses plain pytest and Django's test `Client`, not Django's test runner, which would create a test database of its own. Synchronous tests call the SDK's async methods through `asgiref.sync.async_to_sync`, as synchronous Django views do: ```python from asgiref.sync import async_to_sync from storage.resources import db def test_note_row(note_title): note = Note.objects.create(title=note_title) row = async_to_sync( db.query("SELECT title FROM storage_note WHERE id = $1", note.pk).one )() assert row == {"title": note_title} ``` --- Source: https://encore.dev/docs/py/observability/logging Section: Building (Encore.py) # Logging > Structured logs with Python's standard logging module Encore.py works with Python's standard `logging` module. Install Encore's `ConsoleHandler`, and each log record is written as one structured line in Encore's log format, with its fields and the IDs of the trace and span it was logged in. Encore collects these lines, so you can see each one with the request or message it belongs to. ## Setting up logging Add `ConsoleHandler` to the root logger when your app starts: ```python -- main.py -- import logging from encore.logs import ConsoleHandler logging.basicConfig(level=logging.INFO, handlers=[ConsoleHandler()]) ``` Then log as usual, with a logger per module: ```python import logging log = logging.getLogger(__name__) log.info("order created", extra={"order_id": order.id, "total": 14900, "currency": "SEK"}) ``` `ConsoleHandler` writes to stderr by default. It takes an optional `stream` and `level`: ```python ConsoleHandler(level=logging.WARNING) ``` It writes each record as one JSON line, for example: ```json {"time":"2026-10-19T09:41:07.123456Z","level":"info","message":"order created","logger":"orders.service","trace_id":"4bf92f3577b34da6a3ce929d0e0e4736","span_id":"00f067aa0ba902b7","sampled":true,"code.file.path":"/app/orders/service.py","code.function.name":"create_order","code.line.number":42,"currency":"SEK","order_id":"ord_123","total":14900} ``` Each line has the time, level, message, logger name, where it was logged from (`code.*`), the `extra` fields, and the trace and span IDs of the current span. `ConsoleHandler` ignores logging formatters: the line's format is Encore's. Output from `print`, and lines written by other handlers, is still collected, but as plain text without fields or trace IDs. ## Structured fields Pass variable data as fields with `extra` rather than formatting it into the message: ```python # Prefer log.info("order created", extra={"order_id": order_id, "customer_id": customer_id}) # Avoid log.info(f"order {order_id} created for customer {customer_id}") ``` A message that's the same for every equivalent event can be grouped and counted, and fields can be searched and filtered without parsing the message. Field values are written as JSON. Strings, numbers, booleans, `None`, lists and dicts keep their shape; any other value, such as a `Decimal` or a `datetime`, is written as its `str()`. Python's `logging` refuses `extra` keys that clash with `LogRecord` attributes, such as `name`, `message` or `args`, so pick field names that don't. ## Logging exceptions Log an exception with `log.exception(...)` inside an `except` block, or with `exc_info=True`: ```python try: await charge_customer(customer_id, amount) except PaymentError: log.exception("payment authorization failed", extra={"customer_id": customer_id}) raise ``` The line records the exception's type, message and stack trace as `exception.type`, `exception.message` and `exception.stacktrace`. Formatting the exception into the message with `str(err)` loses the type and stack trace. Log a failure once, at the layer that decides whether to retry, fall back or give up. Lower layers should let the exception propagate rather than logging it and raising it again. ## Trace and span IDs A record logged while an OpenTelemetry span is current carries that span's trace and span IDs. This is what ties a log line to its request in Encore's traces. With [tracing set up](https://encore.dev/docs/py/observability/tracing), records logged while your web framework handles a request carry the request's span, and records logged inside your own spans carry those. The span is read when the record is handled, so set up tracing as well as logging to get the IDs. Records logged outside any span have no trace IDs. ## Encore's own log lines The Encore runtime logs under the `encore.runtime` logger, so its lines follow your logging configuration: with `ConsoleHandler` on the root logger, they're written in the same format as your own, and your log level filters them like any other logger's. ## Log levels Python's levels map onto Encore's: `DEBUG`, `INFO`, `WARNING`, `ERROR` and `CRITICAL`. Set the minimum level with `logging.basicConfig(level=...)`, or per logger with `setLevel`. - `ERROR` is for an operation that failed unexpectedly and couldn't complete. Expected outcomes, such as a declined payment or failed validation, are not errors. - `WARNING` is for something unexpected that the app recovered from, such as falling back to cached data or succeeding on retry. - `INFO` is for meaningful, relatively low-volume events, such as a state change or a completed job. - `DEBUG` is for detail that helps investigate behavior but isn't needed to operate the app. ## What to log Log what the trace doesn't already show. Encore traces database queries, Redis commands, object storage operations and Pub/Sub messages, and your web framework's instrumentation traces requests. Lines like "querying database" or "request completed" duplicate the trace. Do log business events and decisions, with the identifiers needed to investigate them: an order being rejected and why, a fallback being used, a batch completing with its counts. When you only need to know how often something happens, use a [metric](https://encore.dev/docs/py/observability/metrics) instead. ## Sensitive data Encore writes log fields as they are, without redacting them. Never log passwords, API keys, tokens, session IDs, authorization headers, secret values or payment details, and avoid logging whole request or response objects, which often contain them. Prefer internal identifiers, such as a user ID, over personal data, such as an email address. ## Viewing logs `encore run` shows your services' logs as they happen, grouped with the request or message that caused them. `encore logs` shows the logs of a run afterwards, from the app's local telemetry store, so the app doesn't need to be running: ```shell $ encore logs # the latest session $ encore logs --svc orders # only entries involving the orders service $ encore logs --since 10m # only entries that began in the last 10 minutes $ encore logs --errors # only entries in which something failed ``` --- Source: https://encore.dev/docs/py/observability/metrics Section: Building (Encore.py) # Metrics > Track custom metrics in your Python application Encore.py has built-in support for custom metrics: counters, up-down counters, gauges and histograms. You declare a metric in code and record to it where the thing it measures happens. Encore exports it with the rest of your app's telemetry, and you can view it locally with `encore metrics` and in the Encore Cloud dashboard. See the [platform metrics documentation](https://encore.dev/docs/platform/observability/metrics) for exporting metrics to third-party services such as Grafana Cloud and Datadog. ## Defining a metric Declare a metric as a module-level variable, then call its methods to record: ```python -- orders/metrics.py -- from encore import metrics orders_placed = metrics.Counter("orders_placed", description="Orders placed.") ``` ```python -- orders/service.py -- from orders.metrics import orders_placed async def place_order(order) -> None: ... orders_placed.increment() ``` ## Metric types | Type | Measures | Methods | | --- | --- | --- | | `Counter` | A total that only goes up, such as requests handled or bytes processed. | `increment()`, `add(amount)` | | `UpDownCounter` | A total that goes up and down, such as requests in flight or jobs queued. | `increment()`, `decrement()`, `add(amount)` | | `Gauge` | What something is right now, such as a temperature or a pool's size. | `set(value)` | | `Histogram` | How values are distributed, such as request durations or upload sizes. | `record(value)` | ```python from encore import metrics orders = metrics.Counter("orders_placed", description="Orders placed.") queued = metrics.UpDownCounter("jobs_queued", unit="{job}") oven = metrics.Gauge("oven_temperature", unit="Cel") latency = metrics.Histogram("order_latency", unit="s", buckets=[0.1, 1, 10]) orders.increment() queued.add(12) oven.set(180.5) latency.record(0.25) ``` A counter only goes up: `add` ignores negative amounts. Its value resets when the process restarts. Every metric takes these keyword-only options: - `unit`: the unit, as it should be exported. [UCUM](https://ucum.org/) is the convention: `"s"`, `"ms"`, `"By"`, `"{request}"`. - `description`: what the metric measures. - `buckets` (histograms only): the upper bounds of the histogram's buckets, strictly increasing. A value is counted in the first bucket it's no greater than, or in the overflow bucket after the last. Without `buckets`, OpenTelemetry's default bounds are used. ## Rules for declaring metrics Encore reads metric declarations from the source, so: - Declare metrics at module level. - The name must be a string literal: a letter, then letters, digits, `_`, `.`, `-` and `/`, at most 255 characters. - Pass the name as the only positional argument, and `unit`, `description` and `buckets` as keyword arguments with literal values: `buckets=[0.1, 1, 10]`, not a variable. - Use a metric by calling its methods. Encore records which services record to which metrics from these calls, so it rejects passing the metric itself around. Use a [reference](#passing-metrics-to-other-code) instead. ## Scopes The class you declare a metric with decides who records to it, and how it's named when exported: | Classes | Belongs to | Exported as | | --- | --- | --- | | `Counter`, `UpDownCounter`, `Gauge`, `Histogram` | The service that declares it, which alone records to it. Must be declared inside a service. | `.` | | `SharedCounter`, `SharedUpDownCounter`, `SharedGauge`, `SharedHistogram` | The app, shared between services. Encore records each service that records to it. | `` | | `GlobalCounter`, `GlobalUpDownCounter`, `GlobalGauge`, `GlobalHistogram` | The whole app. Any code may record to it, including libraries outside every service, and who does isn't recorded. | `` | For example, `metrics.Counter("orders_placed")` declared in the `shop` service is exported as `shop.orders_placed`. A service's metric names are unique within the service; shared and global metric names are unique within the app. A name can't be one of the metrics Encore records itself, such as `http.server.request.duration`. ```python from encore import metrics # Shared between the services that record to it. cache_hits = metrics.SharedCounter("cache.hits", unit="{hit}") ``` Declare each shared or global metric once in the app. Code in other modules records to it by importing the declaration. ### Binding a metric declared elsewhere To record to a shared or global metric declared in another language, or one you don't want to import, bind it by name with `named` on the class of its kind: ```python from encore import metrics cache_hits = metrics.Counter.named("cache.hits") ``` For a group, give its label type too: `metrics.CounterGroup[Outcome].named("responses_total")`. `named` takes the metric's name alone. The metric keeps the unit, description and buckets of its declaration, and its name isn't prefixed with a service's. Like a declaration, the binding must be a module-level variable and the name a string literal, and Encore checks that a shared or global metric of that kind and labels exists. `named` is available on `Counter`, `UpDownCounter`, `Gauge`, `Histogram` and their groups. ## Labels A group records one series for each combination of its labels. Define the labels as a `TypedDict` or a dataclass, and declare the metric with the group form of its class, `CounterGroup`, `UpDownCounterGroup`, `GaugeGroup` or `HistogramGroup` (or their `Shared…` and `Global…` versions): ```python from typing import TypedDict from encore import metrics class Outcome(TypedDict): method: str status: int requests_handled = metrics.CounterGroup[Outcome]("requests_handled") requests_handled.with_labels({"method": "GET", "status": 200}).increment() ``` Or with a dataclass: ```python from dataclasses import dataclass from encore import metrics @dataclass(frozen=True) class Route: route: str request_latency = metrics.HistogramGroup[Route]("request_latency", unit="s") request_latency.with_labels(Route("/orders")).record(0.025) ``` `with_labels` returns the metric for those labels, with the same methods as an unlabeled metric. Keep it in a variable where you record to it often. - Name the label type directly in the subscript, `CounterGroup[Outcome]`, so Encore can read it. - Label values are `str`, `int` and `bool`. Any other value raises `TypeError`. - Label keys are exported in snake_case: a key `routeName` becomes `route_name`. > **Important** > > Every distinct combination of label values is a time series of its own, tracked in memory and stored by the monitoring system. Use labels for a handful of values, such as a status or a region, never for something like a user ID. Keep the number of series per metric to tens or hundreds, not thousands. ## Passing metrics to other code To hand a metric to other code, pass a reference to it from `ref()`. The code it's handed to records to the metric through it: ```python from encore import metrics signups = metrics.Counter("signups") def count(counter: metrics.Counter) -> None: counter.increment() count(signups.ref()) ``` `ref()` works on every metric and group. ## OpenTelemetry instruments Encore's metrics are instruments of the OpenTelemetry Metrics API. You can also record with the OpenTelemetry API directly, for example for observable instruments, or attributes that labels can't express. Instruments made from Encore's meter provider are exported alongside Encore's metrics: ```python import asyncio from opentelemetry.metrics import CallbackOptions, Observation from encore.metrics import meter_provider job_queue: asyncio.Queue = asyncio.Queue() meter = meter_provider().get_meter(__name__) batch_size = meter.create_histogram("users.batch.size", unit="{user}") batch_size.record(25) def queue_depth(options: CallbackOptions): yield Observation(job_queue.qsize()) meter.create_observable_gauge("jobs.queue.depth", callbacks=[queue_depth]) ``` To have instrumentation packages record through Encore too, install its provider as the OpenTelemetry global, as described in [Tracing](https://encore.dev/docs/py/observability/tracing#setting-up-opentelemetry): ```python from opentelemetry import metrics from encore.metrics import meter_provider metrics.set_meter_provider(meter_provider()) ``` Instruments made with the OpenTelemetry API are named exactly as you create them, without a service prefix, and Encore's parser doesn't see them: the declaration rules above don't apply. A measurement recorded while a sampled span is current carries that span as an exemplar, linking it to the trace. ## Viewing metrics Locally, `encore metrics` shows the metrics your services reported in the last hour: ```shell $ encore metrics list # every metric, with stats over the last 5 minutes $ encore metrics list --watch # refresh every few seconds $ encore metrics get orders.orders_placed # one metric's series $ encore metrics get orders.requests_handled --attr status=500 ``` Under `encore test`, metrics are recorded as usual but not exported. --- Source: https://encore.dev/docs/py/observability/tracing Section: Building (Encore.py) # Tracing > See what your app does, request by request Encore records [distributed traces](https://encore.dev/docs/tracing) of what your app does. Encore.py traces the calls your code makes to Encore's infrastructure on its own. Requests to your web framework and your own operations are added through OpenTelemetry: you install Encore's tracer provider, instrument your framework with the standard OpenTelemetry package for it, and every span lands in the same trace. ## What Encore traces automatically Without any setup, Encore records a span for: - SQL queries and transactions - Redis commands - Object storage operations - Pub/Sub publishes - The processing of each Pub/Sub message, as a trace of its own that links to the publish that sent it Apart from message processing, each of these spans is a child of the OpenTelemetry span current in your code when you make the call. Once your framework is instrumented, a database query made while handling a request appears inside that request's trace. ## Setting up OpenTelemetry Encore records its own spans and metrics through its own providers, and doesn't install the OpenTelemetry globals, so it leaves any setup of your own alone. To send your app's spans, and those of instrumentation packages, to Encore, install Encore's providers as the globals when your app starts: ```python -- main.py -- from opentelemetry import metrics, trace from encore.metrics import meter_provider from encore.tracing import tracer_provider trace.set_tracer_provider(tracer_provider()) metrics.set_meter_provider(meter_provider()) ``` Do this before you instrument your framework or create tracers. `tracer_provider()` and `meter_provider()` return the same provider every time you call them. You can also pass the provider to a single instrumentation instead of installing it globally, for example `FastAPIInstrumentor.instrument_app(app, tracer_provider=tracer_provider())`. ## Instrumenting your web framework HTTP requests are traced by your framework's OpenTelemetry instrumentation package. For FastAPI, add `opentelemetry-instrumentation-fastapi` to your dependencies and instrument the app: ```python -- main.py -- from contextlib import asynccontextmanager import encore from encore.metrics import meter_provider from encore.tracing import tracer_provider from fastapi import FastAPI from opentelemetry import metrics, trace from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor trace.set_tracer_provider(tracer_provider()) metrics.set_meter_provider(meter_provider()) @asynccontextmanager async def lifespan(app: FastAPI): await encore.start() yield app = FastAPI(lifespan=lifespan) FastAPIInstrumentor.instrument_app(app) ``` For Django, use `opentelemetry-instrumentation-django`: ```python from opentelemetry.instrumentation.django import DjangoInstrumentor DjangoInstrumentor().instrument() ``` Other frameworks and HTTP clients have instrumentation packages of their own, such as `opentelemetry-instrumentation-flask` and `opentelemetry-instrumentation-httpx`. Instrumenting your HTTP client propagates the trace to the services you call. Importing `encore` sets the `OTEL_SEMCONV_STABILITY_OPT_IN` environment variable to `http`, unless you've set it, so instrumentation packages record the stable HTTP semantic conventions that Encore expects. Instrumentation packages read the variable when they're imported, so import `encore` first. ## Adding your own spans Create spans with the standard OpenTelemetry API: ```python from opentelemetry import trace from orders.db import db tracer = trace.get_tracer(__name__) async def get_user(user_id: int): with tracer.start_as_current_span("load user") as span: span.set_attribute("user.id", user_id) return await db.query("SELECT * FROM users WHERE id = $1", user_id).one() ``` The query's span is recorded as a child of `load user`, which in turn is a child of the request's span. `start_as_current_span` also works as a decorator, including on `async` functions: ```python @tracer.start_as_current_span("recalculate prices") async def recalculate_prices() -> None: ... ``` Spans support attributes, events, links and status as usual. An exception that leaves a `start_as_current_span` block is recorded on the span as an event, and sets the span's status to error. The current span follows your code across `await` and into tasks it starts with `asyncio.create_task` or `asyncio.gather`. Encore decides which traces are sampled. A span in a trace that isn't sampled doesn't record anything, so its `is_recording()` returns `False`. > **Note** > > Inside a Pub/Sub subscription handler, the message's processing span isn't yet made the current span in Python. The handler's own spans, and the calls it makes, aren't linked to the message's trace yet. ## Logs in traces With Encore's [`ConsoleHandler`](https://encore.dev/docs/py/observability/logging) on your loggers, each log record carries the trace and span IDs of the span current when it's logged, which ties it to its place in the trace. ## Viewing traces Under `encore run`, traces are captured locally. View them in the local development dashboard, whose address `encore run` prints, or from the command line: ```shell $ encore trace list # the newest traces, most recent first $ encore trace list --limit 10 $ encore trace get # every event of one trace ``` For deployed environments, view traces in the [Encore Cloud dashboard](https://app.encore.dev). See [Distributed Tracing](https://encore.dev/docs/tracing) for more on what traces show. --- Source: https://encore.dev/docs/py/develop/config Section: Building (Encore.py) # Configuration > Typed settings that vary by environment A config is a typed value your code reads, such as limits, timeouts and feature flags. You declare its type in Python and write its value in a TOML or JSON file beside the code, with overlays that change it for some environments only. Encore checks every value against the type before your service starts, and under `encore run` you can change a value without restarting anything. > **Important** > > For sensitive data, use [secrets](https://encore.dev/docs/py/primitives/secrets) instead. A config can [read secrets](#secrets-in-config) but never holds them in its files. ## Declaring a config Describe the config's value with a dataclass, and declare the config as a module-level variable: ```python -- orders/settings.py -- from dataclasses import dataclass, field from datetime import timedelta from typing import Annotated from encore.config import Config, Max, Min @dataclass class Checkout: max_items: Annotated[int, Min(1), Max(500)] currencies: list[str] = field(default_factory=lambda: ["USD"]) payment_timeout: timedelta = timedelta(seconds=90) checkout = Config[Checkout]("checkout") ``` Then write its value in a file named after it, beside the service's `encore.service.json`: ```toml -- orders/checkout.config.toml -- max_items = 100 payment_timeout = "2m" ``` Read the value with `.value`. It's an instance of your type: ```python from orders.settings import checkout def validate_cart(items: list[str]) -> None: limit = checkout.value.max_items if len(items) > limit: raise ValueError(f"at most {limit} items per order") ``` Encore reads config declarations from the source, so: - Declare configs at module level. - The name must be a string literal and an identifier: a letter or underscore, then letters, digits and underscores. - The type argument is required: `Config[Checkout]("checkout")`. Without it, reading `.value` raises `TypeError`. - Use the config by reading `.value` or calling `.on_change()`. Encore records which services read which configs from these calls, so it rejects passing the config object itself around. ## Supported types The config's type is a dataclass, a `TypedDict` or an `Enum`, built from: - `str`, `int`, `float`, `bool` - `Literal[...]`, `Optional[...]`, `Enum` subclasses - `list[...]` and `dict[str, ...]` - nested dataclasses and `TypedDict`s, including generic ones - `typing.Any` - `datetime.timedelta`, written as a duration string such as `"30s"`, `"250ms"` or `"1h30m"` - [`Secret`](#secrets-in-config) fields A dataclass field with a default, or an `Optional` field, may be left out of the files. A union of dataclasses is supported when each has a `Literal` tag field that tells them apart. Encore reports a type it can't describe as an error at the part of the type that's the problem. ### Constraints Narrow what a field accepts with `Annotated` markers from `encore.config`: | Marker | Applies to | Example | | --- | --- | --- | | `Min(n)`, `Max(n)` | `int`, `float` | `Annotated[int, Min(1), Max(500)]` | | `MinLen(n)`, `MaxLen(n)` | `str` | `Annotated[str, MaxLen(64)]` | | `Pattern(regex)` | `str` | `Annotated[str, Pattern("^[A-Z]+$")]` | | `MinItems(n)`, `MaxItems(n)` | `list` | `Annotated[list[str], MinItems(1)]` | Encore checks values against these before the service sees them. At run time the markers do nothing. ## Config scopes As with secrets, the class you declare a config with decides who it belongs to: | Class | Belongs to | Its files live | | --- | --- | --- | | `Config[T]` | The service that declares it, which has its own value. Must be declared inside a service. | Beside the service's `encore.service.json` | | `SharedConfig[T]` | The whole app, with one value. Encore records each service that reads it. | Beside `encore.app` | | `GlobalConfig[T]` | The whole app, with one value, available to every service. Reads are not recorded. | Beside `encore.app` | ```python from dataclasses import dataclass from encore.config import SharedConfig @dataclass class Flags: new_checkout: bool = False flags = SharedConfig[Flags]("flags") ``` ## Environment-specific values A config's value is merged from several files. Each layer overrides the ones before it, using [JSON Merge Patch](https://www.rfc-editor.org/rfc/rfc7396) semantics: 1. The defaults in your type. 2. `.config.toml`: the value in every environment. 3. `..config.toml`: environments of one type, where the type is `local` (`encore run`), `test` (`encore test`), `development`, `preview` or `production`. 4. `.@.config.toml`: the environment named ``. 5. `.user.config.toml`: your own overrides, applied to your `encore run` only. These files are gitignored (`*.user.config.*`) and never read in tests. Each file can be `.toml` or `.json`. For example: ```toml -- orders/checkout.config.toml -- max_items = 100 ``` ```toml -- orders/checkout.production.config.toml -- max_items = 500 payment_timeout = "30s" ``` ```toml -- orders/checkout.@prod-eu.config.toml -- currencies = ["EUR", "SEK"] ``` Each file is checked as a partial value, so a mistake is reported in the file that makes it, and the merged value is then checked in full. A file whose name doesn't match a declared config is ignored, unless its name is close to one, which is reported as a likely typo. ## Reacting to changes Under `encore run`, editing a config file updates the value in running services without a rebuild or restart. An invalid edit is reported, and services keep the last valid value. Each read of `.value` returns the newest value. Keep the result in a local variable when you want one consistent value for the duration of an operation: ```python settings = checkout.value # one snapshot for this whole request ``` To act when the value changes, register a callback with `on_change`. It runs on a background thread, so keep it quick. It returns a function that stops the calls: ```python import logging log = logging.getLogger(__name__) def log_change(value: Checkout) -> None: log.info("checkout config changed", extra={"max_items": value.max_items}) stop = checkout.on_change(log_change) ``` An exception raised by the callback is logged, and later changes still reach it. ## Secrets in config A field of type `Secret[str]` or `Secret[bytes]` reads a secret instead of holding a value. Its default names the secret. The secret must also be declared in code, with the config's scope: a service's own `Secret` for a `Config`, a `SharedSecret` for a `SharedConfig`, and a `GlobalSecret` for a `GlobalConfig`. ```python -- orders/settings.py -- from dataclasses import dataclass from encore.config import Config from encore.secrets import Secret stripe_key = Secret[str]("StripeKey") @dataclass class Payments: provider_url: str api_key: Secret[str] = Secret[str]("StripeKey") payments = Config[Payments]("payments") def auth_header() -> str: return f"Bearer {payments.value.api_key.value}" ``` The field's type is always `Secret`, whatever the config's scope. Secret fields can't be written in config files, and can't be `Optional` or be inside lists, dicts or unions. ## Inspecting values with the CLI The `encore config` commands show and edit config values: ```shell $ encore config list # the app's configs, and whether each value is valid $ encore config show orders.checkout # the merged value, and which file each part comes from $ encore config show orders.checkout --env-type production $ encore config validate # check every file, for every environment type $ encore config schema orders.checkout # the JSON Schema Encore derived from the type ``` Name a service's config as `.`, and a shared or global one by its name alone. `encore config set` and `encore config unset` change one value in one file, keeping the rest of the file as it was. They write your user override by default; pass `--base`, `--env-type ` or `--env ` to change another layer: ```shell $ encore config set orders.checkout max_items 20 $ encore config set orders.checkout max_items 500 --env-type production $ encore config unset orders.checkout max_items ``` ## Editor support Run `encore config link` to name each config file's schema in the file: a `#:schema` comment in TOML, or a `"$schema"` key in JSON. Editors with JSON Schema support then check and complete what you write. The schemas are served by the local Encore daemon. Run it again after adding a file. ## Testing Under `encore test`, a config has the value its `test` overlay gives it: `.test.config.toml` over the base file. User overrides are never applied in tests. --- Source: https://encore.dev/docs/py/develop/shutdown Section: Building (Encore.py) # Graceful shutdown > Drain work and release resources when a service stops When a service is stopped, for example during a deploy or when `encore run` restarts it, it receives `SIGTERM`. Encore then shuts down gracefully: it stops taking new Pub/Sub messages, waits for message handlers in progress, closes database, Redis and object storage connections, and flushes metrics and traces. Most apps need no code for this. Read on if your app runs its own background work that uses Encore resources, or if you want to control when Encore shuts down. ## Default behavior Importing `encore` sets up shutdown for you: - It registers an exit hook that shuts Encore down when the interpreter exits. - It installs a `SIGTERM` handler that exits the interpreter, with exit code `0` if shutdown completed within its deadline and `1` otherwise. It does this only when the process has no `SIGTERM` handler yet at import time, and not under `encore test`, where the test runner handles the signal. This fits servers like Uvicorn, which handle `SIGTERM` themselves: Uvicorn stops accepting connections and drains requests in progress, then restores the previous signal handler and raises the signal again. That hands over to Encore, which drains Pub/Sub and cleans up after your HTTP server has finished. A FastAPI app run with `uvicorn.run(app, ...)` shuts down gracefully without any shutdown code. If your server handles `SIGTERM` without raising it again, Encore still shuts down from its exit hook as the interpreter exits. ## How shutdown proceeds Shutdown moves through these phases, which `shutdown.phase()` returns as a `ShutdownPhase`: | Phase | Meaning | | --- | --- | | `NOT_STARTED` | Shutdown hasn't started. | | `STARTED` | Pub/Sub has stopped taking new messages. Encore waits for message handlers in progress, and in [cooperative mode](#cooperative-shutdown) for your application to signal that it has drained. Your HTTP server is still yours to stop. | | `CLEANUP` | Encore closes SQL, Redis and object storage connections, then exports the final metrics and flushes traces. | | `COMPLETED` | Cleanup has finished, whether it succeeded or not. | Once cleanup has run, Encore's resources can't be used anymore, even through objects you still hold. New operations fail, interrupted object downloads raise an error rather than returning partial data, and Redis subscriptions end. Clients you create yourself from a resource's connection string are yours to close. ## Shutting down from your own code To shut Encore down at a point you choose, call `shutdown.begin()` and await `shutdown.completed()`. For example, in a FastAPI lifespan handler, the code after `yield` runs once Uvicorn has drained HTTP requests: ```python -- main.py -- import logging from contextlib import asynccontextmanager import encore from encore import shutdown from fastapi import FastAPI log = logging.getLogger(__name__) @asynccontextmanager async def lifespan(app: FastAPI): await encore.start() yield # HTTP requests have drained: shut Encore down and wait for it. shutdown.begin() result = await shutdown.completed() log.info("encore shut down", extra={"result": result.value}) app = FastAPI(lifespan=lifespan) ``` `begin()` returns immediately, and all callers share one shutdown: repeated calls don't restart it or reset its deadline. When the interpreter exits afterwards, Encore's own hooks see that shutdown has already completed. ## Cooperative shutdown Use cooperative shutdown when work your application started, such as background tasks, may still be using Encore resources when shutdown begins. Encore then stops Pub/Sub right away, but doesn't close resources until you call `mark_application_drained()` or the [deadline](#deadline-and-forced-cleanup) expires. For example, a worker process with its own background tasks: ```python -- worker.py -- import asyncio import signal import encore from encore import shutdown async def process_jobs(stop: asyncio.Event) -> None: while not stop.is_set(): ... # take a job and process it, using Encore resources async def main() -> int: await encore.start() stop = asyncio.Event() asyncio.get_running_loop().add_signal_handler(signal.SIGTERM, stop.set) workers = [asyncio.create_task(process_jobs(stop)) for _ in range(4)] await stop.wait() shutdown.begin(cooperative=True) # stop Pub/Sub, keep resources open await asyncio.gather(*workers) # let the jobs in progress finish shutdown.mark_application_drained() result = await shutdown.completed() return result.code() if __name__ == "__main__": raise SystemExit(asyncio.run(main())) ``` Keep these rules in mind: - Call `begin()` before awaiting `started()`, `cleanup()` or `completed()`. Awaiting them before `begin()` raises `ShutdownError`. - The first call to `begin()` chooses the mode. Later calls don't change it. - In cooperative mode, call `mark_application_drained()` before awaiting `cleanup()` or `completed()`. Otherwise Encore waits until the deadline expires. - Keep the event loop running until `completed()` returns. Cancelling a wait doesn't cancel the shutdown itself. `await shutdown.started()` and `await shutdown.cleanup()` wait until shutdown has reached or passed those phases, and return immediately if it already has. You don't need to await every phase in turn. ## Deadline and forced cleanup Shutdown has a deadline, five seconds by default, which Encore sets for each deployment. It starts when shutdown begins and covers both draining and cleanup. When it expires: 1. The result becomes `TIMEOUT`, and stays that way even if cleanup later succeeds. 2. If Encore is still waiting for draining, it stops waiting and starts cleanup, even without `mark_application_drained()`. 3. Cleanup continues, and `completed()` returns once it has finished. The deadline doesn't stop the process, and `completed()` can return after it. Your host or server decides when to terminate the process. To start cleanup before the deadline without waiting for draining, call `force_cleanup()`: ```python shutdown.force_cleanup() # requires begin() first await shutdown.cleanup() # cleanup has started result = await shutdown.completed() # cleanup has finished ``` `force_cleanup()` returns immediately and can be called repeatedly. Forcing cleanup doesn't make the result `TIMEOUT`. Neither the deadline nor `force_cleanup()` stops handlers that are still running. They may continue and fail as the resources they use close, so drain your own work before `mark_application_drained()` where you can. ## Results and errors `completed()` returns a `ShutdownResult`: | Result | Meaning | `code()` | | --- | --- | --- | | `ShutdownResult.SUCCESS` | Cleanup finished before the deadline. | `0` | | `ShutdownResult.TIMEOUT` | The deadline expired before cleanup finished. Cleanup has still finished when the result is returned, but a timeout doesn't mean every step succeeded. | `1` | If cleanup fails, or shutdown's progress can't be observed, `completed()` raises `ShutdownError`. `code()` gives you a process exit code. --- Source: https://encore.dev/docs/py/develop/policies Section: Building (Encore.py) # Policies > Rules about who may use what, checked against your code A policy is a rule about your app that Encore checks every time it reads your code. "Only services in `billing/payments` may write to databases holding card data." "A service may not call into another domain's services." "Every public endpoint must be exposed on the `api` gateway." You write the rule once, and Encore checks every service, resource and use of a resource against it, including the ones added after the rule was written. Policies are written in [CEL](https://cel.dev), a small expression language, over a model of your app that Encore builds from your code: its services, resources and endpoints, which services use which resources and how, and which [domains](#domains) and [tags](#tags) everything belongs to. They live in your `encore.app.json`, or in the `encore.domain.json` of a domain. > **Important** > > A broken policy never stops `encore run`. Your app keeps running, and Encore reports what its policies find next to it: in the terminal, in `encore policy check`, and to AI agents through the MCP server. ## An example A payments team keeps card data in a database tagged `data: pci`, and wants only its own services to write to it: ```json -- encore.app.json -- { "id": "shop", "policies": [ { "name": "pci-writes", "subjects": ["resource_link"], "match": "self.resource.effective_tags.data == 'pci' && self.access >= Access.WRITE", "require": "within('billing/payments', self.service.domain)", "message": "{self.service.name} writes PCI data in {self.resource.name}" } ] } ``` The policy applies to every `resource_link`, which is one service's use of one resource. `match` picks the uses it is about: writes to anything tagged `data: pci`. `require` says what each of those must make true: the service doing the writing is in the `billing/payments` domain. When a `reports` service, which is in no domain, starts writing to the `ledger` database, Encore reports: ``` error[ENC0061]: policy violation in `pci-writes`: reports writes PCI data in ledger --> reports/api.py:5:11 | 5 | await ledger.exec("update entries set amount = 0") | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ reports writes ledger here | ::: encore.app.json:8:46 | 8 | "require": "within('billing/payments', self.service.domain)", | ------------------- no domain | ::: reports/encore.service.json:1:11 | 1 | { "name": "reports", "lang": "python" } | --------- reports is in no domain ``` It points at the line of code that breaks the policy, underlines the part of `require` that came out false with its value, and shows where that value comes from. ## Domains A domain is a part of your app with a name: a directory with an `encore.domain.json` in it, and everything below that directory. Services, resources, frontends and gateways belong to the domain around the directory they are declared in. ```json -- billing/payments/encore.domain.json -- { "name": "payments", "description": "Card payments and the ledger.", "tags": { "data": "pci" } } ``` | Field | Required | Description | |---|---|---| | `name` | Yes | The domain's name, unique among its siblings. Lowercase letters, digits and `-`. | | `description` | No | What the domain is for. | | `tags` | No | The domain's [tags](#tags), which everything in it gets. | | `policies` | No | The domain's own [policies](#a-domain-s-policies). | Domains nest. A domain's id is its name, after the names of the domains it is in: `billing/payments` for a `payments` domain inside a `billing` domain. The id comes from the names, not the directory names, so the `payments` domain above could be in a directory called anything. Your app itself is the root domain, with the id `""`. Everything that is in no declared domain is in the root, so every service and resource always has a domain. A domain can't be declared inside a service's directory, though a service directory can be the root of one. Domains don't change how your app builds or runs: they exist for policies, and for Encore's tools to group your app by. ## Tags A tag is a key and a value, like `data: pci` or `team: payments`, that says something about a part of your app. Domains, services, frontends, gateways and resources can all have tags. A domain's tags go in its `encore.domain.json`, and a service's in its `encore.service.json`: ```json -- reports/encore.service.json -- { "name": "reports", "lang": "python", "tags": { "team": "analytics" } } ``` A resource's tags go in its declaration: ```python -- billing/payments/vault/db.py -- from encore.storage import postgres ledger = postgres.Database("ledger", tags={"data": "pci", "tier": "core"}) ``` Every resource declaration takes `tags=`: databases, buckets, topics, subscriptions, Redis databases, cron jobs, secrets, configs, metrics and context fields. The dict must be written out in the declaration, as a literal, because Encore reads it from the code without running it. Keys are lowercase letters, digits, `_` and `-`, start with a letter, and are at most 63 characters long. Values follow the same rules but may be empty. Something can have at most 32 tags, and keys starting with `encore` are reserved. Each part of your app has two sets of tags in the policy model: - `tags`: its own, as declared. - `effective_tags`: the tags of the domains it is in, merged from the outermost to the innermost, then its own. The nearer one wins when two set the same key. The `ledger` database above has `data: pci` among its effective tags because its domain sets it, without declaring it itself. Policies usually read `effective_tags`. A tag something doesn't have reads as the empty string, so `self.tags.data == 'pci'` is simply false for something with no `data` tag. To test whether a tag is set at all, use `has(self.tags.data)`. ## Writing a policy A policy is an object in the `policies` list of `encore.app.json` or of an `encore.domain.json`: | Field | Required | Description | |---|---|---| | `name` | Yes | Unique among the file's policies. Lowercase letters, digits and `-`, starting with a letter. | | `subjects` | Yes | The kinds of thing the policy judges. See [Subjects](#subjects). | | `match` | No | A CEL expression choosing which of those things the policy applies to. Without it, the policy applies to all of them. | | `require` | Yes | A CEL expression each thing the policy applies to must make true. | | `effect` | No | `"deny"`, the default, or `"warn"`. See [Effects](#effects). | | `message` | No | What a violation says. See [Messages](#messages). | Encore checks a policy when it reads it, before judging anything by it. A field the subject doesn't have, a misspelled function, comparing an enum with a string, or an expression that isn't true or false is reported as `ENC0060`, with a suggestion where there is one, and the policy is left out until it is fixed. ### Subjects A policy judges one or more kinds of subject, each in turn as `self`: | Subject | `self` is | |---|---| | `domain` | A domain, the root among them. | | `service` | A service. | | `frontend` | A frontend. | | `gateway` | A gateway. | | `resource` | A resource: a database, bucket, topic, subscription, Redis database, cron job, secret, config, metric or context field. | | `endpoint` | An endpoint of a service. | | `resource_link` | One service's use of one resource: every use in its code, together. | | `service_link` | One service calling another. | | `subscription_link` | One service subscribing to a topic. | | `exposure_link` | One endpoint exposed on one gateway: reachable from outside the app through it. | A policy can name several kinds of subject, like `["service", "resource"]`. Then `self` has only the fields all of them share, here `name`, `domain`, `tags`, `effective_tags` and `location`. Encore tells you which kinds have a field you use when not all of them do. ### Variables Every expression reads three variables: | Variable | What it is | |---|---| | `self` | The subject being judged. | | `app` | The whole app: `app.services`, `app.resources` and `app.domains` by name or id, `app.endpoints`, and every kind of link. | | `domain` | The domain the policy belongs to: the one whose `encore.domain.json` declares it, or the root for a policy in `encore.app.json`. | ### What a policy can read The most useful fields of each subject: | Subject | Fields | |---|---| | Domain | `id`, `name`, `parent`, `ancestry` (the ids of the domains it is in and its own, the root first), `depth`, `tags`, `effective_tags` | | Service | `name`, `language`, `domain`, `tags`, `effective_tags`, `root_dir`, `workers`, `gateways` (those any of its endpoints is exposed on), `serves_traffic`, `hosts_frontend`, `hosts_gateway` | | Frontend | `name`, `language`, `service` (the one hosting it, if any), `domain`, `tags`, `effective_tags` | | Gateway | `name`, `default`, `service` (the one running its middleware, if any), `domain`, `tags`, `effective_tags`, `origins_with_credentials`, `origins_without_credentials`, `any_origin_with_credentials` | | Resource | `id` (like `postgres_database:ledger`), `kind`, `name`, `domain`, `tags`, `effective_tags`, and its kind's own settings: `bucket.public`, `pubsub_topic.exactly_once`, `pubsub_subscription.max_retries`, `secret.scope`, and so on | | Endpoint | `service`, `name`, `protocol`, `methods`, `path`, `gateways`, `sensitive` | | Resource link | `service`, `resource`, `operations` (what the service does with it), `access` (the highest access those need), `locations`, `crossing` | | Service link | `caller`, `callee`, `locations`, `crossing` | | Subscription link | `service`, `subscription`, `topic`, `crossing` | | Exposure link | `gateway`, `endpoint`, `crossing` | Everything also has a `location`, with the `file`, `line` and `column` it is declared at. Every link has a `crossing`: the domain boundaries it crosses, from the side that acts to the side acted on. For a service in `billing/payments` using a resource in `identity`, `crossing.common` is the root, the innermost domain holding both; `crossing.exits` is `billing/payments` then `billing`; and `crossing.enters` is `identity`. A link inside one domain crosses nothing, so `self.crossing.enters.size() == 0`. Enumerations are written by their type and name: | Enumeration | Values | |---|---| | `Access` | `READ`, `WRITE`, `ADMIN`, in that order, so `self.access >= Access.WRITE` holds for writes and schema changes. | | `Operation` | `POSTGRES_READ`, `POSTGRES_WRITE`, `POSTGRES_ADMIN`, `BUCKET_LIST`, `BUCKET_READ`, `BUCKET_WRITE`, `BUCKET_DELETE`, `BUCKET_SIGNED_UPLOAD_URL`, `TOPIC_PUBLISH`, `SUBSCRIPTION_RECEIVE`, `REDIS_READ`, `REDIS_WRITE`, `SECRET_READ`, `CONFIG_READ`, `METRIC_RECORD`, and more. | | `Resource.Kind` | `POSTGRES_DATABASE`, `BUCKET`, `PUBSUB_TOPIC`, `PUBSUB_SUBSCRIPTION`, `REDIS_DATABASE`, `CRON_JOB`, `SECRET`, `CONFIG`, `METRIC`, `CONTEXT_FIELD` | | `Resource.Scope` | `SERVICE`, `SHARED`, `GLOBAL` | | `Service.Language` | `GO`, `TYPESCRIPT`, `RUST`, `PYTHON` | | `Endpoint.Protocol` | `HTTP`, `GRAPHQL`, `GRPC`, `PASSTHROUGH`, `FRAMEWORK` | `encore policy eval` [shows](#trying-an-expression) any field's value for any subject. ### Functions Policies can use CEL's standard functions, like `size()`, `startsWith()`, `matches()`, `in`, `all()` and `exists()`, and its strings extension. Encore adds three: | Function | What it does | |---|---| | `within(id, d)` | Whether the domain `d` is the domain `id` or is inside it. `within('billing', self.service.domain)` holds for a service in `billing` or `billing/payments`. | | `glob(pattern, s)` | Whether `s` matches `pattern`, where `*` matches any run of characters and `?` any one. `glob('internal-*', self.name)`. | | `reaches(s)` | The resources the service `s` can reach: those it uses, and those reached by the services it calls and by the subscribers of the topics it publishes to. | ### Effects A policy with `"effect": "deny"`, the default, reports what breaks it as an error. `encore policy check` exits with an error when there is one, so a CI step can stop a change that breaks a policy. A policy with `"effect": "warn"` reports what breaks it as a warning, and stops nothing. It is the way to introduce a rule to an app that doesn't keep it yet. ### Messages A violation's message is the policy's `message`, with each `{expression}` in it replaced by the expression's value for the subject that broke it: ```json "message": "{self.service.name} writes PCI data in {self.resource.name}" ``` Strings are written as they are, enum values by their names (`WRITE`), and lists joined by `, `. Write `{{` and `}}` for literal braces. Without a `message`, a violation says what broke the policy, like `reports using database ledger`. ## A domain's policies A domain can declare policies of its own, in its `encore.domain.json`. They judge only what is in the domain, and the links with an end in it: a domain's policy over `service_link` sees calls into the domain from outside as well as calls out of it. In its expressions, `domain` is the domain itself, so the same policy reads the same in any domain: ```json -- billing/encore.domain.json -- { "name": "billing", "policies": [ { "name": "no-calls-out", "subjects": ["service_link"], "match": "within(domain.id, self.caller.domain)", "require": "within(domain.id, self.callee.domain)", "effect": "warn", "message": "{self.caller.name} calls {self.callee.name}, outside {domain.id}" } ] } ``` The app's own policies, in `encore.app.json`, judge everything. ## More examples Only the `api` gateway exposes endpoints to the internet: ```json { "name": "one-gateway", "subjects": ["exposure_link"], "require": "self.gateway.name == 'api'" } ``` No public buckets, outside the `marketing` domain: ```json { "name": "private-buckets", "subjects": ["resource"], "match": "self.kind == Resource.Kind.BUCKET && !within('marketing', self.domain)", "require": "!self.bucket.public" } ``` Every service says which team owns it: ```json { "name": "owned", "subjects": ["service"], "require": "has(self.effective_tags.team)", "effect": "warn", "message": "{self.name} has no team tag" } ``` A service that handles personal data can't reach a database tagged `analytics`: ```json { "name": "pii-stays-out-of-analytics", "subjects": ["service"], "match": "self.effective_tags.data == 'pii'", "require": "!reaches(self).exists(r, r.effective_tags.data == 'analytics')" } ``` Only schema migrations change a database's schema, so no service code needs admin access: ```json { "name": "no-admin", "subjects": ["resource_link"], "require": "self.access < Access.ADMIN" } ``` ## Checking your app `encore run` checks your app against its policies each time it reads your code, and shows what they find under the app's output. In the terminal UI, press `p` to see every finding. With `--json`, each report is an event of kind `policy`. `encore policy check` checks the app once, prints each finding, and exits with an error if a `deny` policy is broken or a policy can't be read: ```shell $ encore policy check ``` `encore policy ls` lists the app's policies, where each is declared, and how many things break it. When a policy can't be decided for something, because an expression fails on it, the policy counts as broken by it. The usual cause is reading a key a map doesn't have, like `app.services['billing']` in an app with no `billing` service: test for it first, with `has()` or `in`. ### Trying an expression `encore policy eval` evaluates one expression, the way a policy's `match` or `require` would, and prints its value: ```shell $ encore policy eval 'self.access' --subject 'resource_link:reports->postgres_database:ledger' "WRITE" (Access) ``` The subject is named the way `encore policy check` names it: `service:billing`, `resource:postgres_database:ledger`, `domain:billing/payments`, `service_link:reports->billing`. Without `--subject`, the expression can read only `app` and `domain`. Use `--domain` to evaluate it as a policy of that domain would be. ### For AI agents Encore's [MCP server](https://encore.dev/docs/mcp) has the tools `policy_check`, `policy_list` and `policy_eval`, which do what the commands above do. Its other tools, like the ones that call an endpoint or run tests, add a warning to their output while a policy is broken, so an agent finds out it broke one without being stopped. --- Source: https://encore.dev/docs/rs Section: Building (Encore.rs) # Encore.rs > An infrastructure SDK for Rust Encore.rs is an infrastructure SDK for Rust. You declare the infrastructure your application needs, such as databases, Pub/Sub topics, object storage buckets, Redis databases, cron jobs and secrets, as `static`s in your Rust code, and use them through their methods. Encore reads the declarations from your source, runs matching infrastructure locally with `encore run`, and provisions the real resources when you deploy. The SDK is the `encore-dev` crate, imported as `encore`. It links Encore's Rust runtime directly: there is no FFI layer and no code generation. Encore.rs covers infrastructure only. It has no API framework: you serve HTTP with the web framework you already use, such as [axum](https://github.com/tokio-rs/axum) or actix-web, and call the SDK from your handlers. > **Note** > > Encore.rs is in early access. APIs may still change, and some features are not yet available. The sections below note where behavior is incomplete. ## What you get with Encore.rs - **Infrastructure as Rust**: declare [databases](https://encore.dev/docs/rs/primitives/databases), [Pub/Sub topics and subscriptions](https://encore.dev/docs/rs/primitives/pubsub), [object storage buckets](https://encore.dev/docs/rs/primitives/object-storage), [Redis databases](https://encore.dev/docs/rs/primitives/redis), [cron jobs](https://encore.dev/docs/rs/primitives/cron-jobs), [secrets](https://encore.dev/docs/rs/primitives/secrets) and [configuration](https://encore.dev/docs/rs/develop/config) as `static` values. Encore works out from your code which service uses what, and grants exactly that access. - **Local infrastructure, automatically**: `encore run` starts Postgres, a Pub/Sub broker, an object storage emulator and a Redis server for your app, applies your migrations, and builds and runs each service with Cargo. - **Isolated tests**: under `encore test`, every test gets its own databases, buckets, Redis databases and topics. See [Testing](https://encore.dev/docs/rs/develop/testing). - **Observability**: database, Pub/Sub, object storage and Redis calls are traced, and your own spans join them through OpenTelemetry. See [Tracing](https://encore.dev/docs/rs/observability/tracing), [Logging](https://encore.dev/docs/rs/observability/logging) and [Metrics](https://encore.dev/docs/rs/observability/metrics). - **Your own web framework**: Encore.rs doesn't own your `main` or your router. ## The infrastructure primitives Each primitive is a module-level `static`, built with a `const` constructor. Options are chained onto the constructor. **Database** ```rust use encore::storage::postgres::Database; static DB: Database = Database::new("orders").migrations("./migrations"); ``` **Pub/Sub** ```rust use encore::pubsub::Topic; use serde::{Deserialize, Serialize}; #[derive(Serialize, Deserialize)] struct OrderPlaced { order_id: String, } static ORDERS: Topic = Topic::new("order-placed"); ``` **Object Storage** ```rust use encore::objects::Bucket; static INVOICES: Bucket = Bucket::new("invoices"); ``` **Redis** ```rust use encore::storage::redis; static SESSIONS: redis::Database = redis::Database::new("sessions"); ``` **Cron Jobs** ```rust use encore::cron::{Job, Schedule}; static SEND_INVOICES: Job = Job::new("send-invoices", "/internal/send-invoices", Schedule::Every("2h")) .title("Send pending invoices"); ``` **Secrets** ```rust use encore::secrets::Secret; static STRIPE_KEY: Secret = Secret::new("StripeKey"); ``` Encore reads these declarations statically, so a few rules apply to all of them: - A declaration is a `static` at module level, not a local variable or a value built in a function. - Names and options are string literals (or other literals) written at the declaration. - A declared resource is used by calling its methods. To hand it to other code, create a handle, such as a `DatabaseRef` from `DB.read()`. - Every use must be inside a [service](https://encore.dev/docs/rs/primitives/services), so Encore knows which service to grant it to. Encore reports a broken rule when it parses the app, on `encore run`, `encore test` or `encore build`, even where the Rust compiler accepts the code. ## Bring your own web framework A Rust service is an ordinary binary. It listens on the address Encore gives it in the `ENCORE_LISTEN_ADDR` environment variable (`PORT` carries the same port), and calls the SDK from its handlers. With axum: ```rust -- src/main.rs -- use axum::{extract::Path, http::StatusCode, routing::post, Router}; use encore::storage::postgres::Database; static DB: Database = Database::new("users").migrations("./migrations"); async fn add_user(Path(name): Path) -> StatusCode { match DB.execute("INSERT INTO users (name) VALUES ($1)", &[&name]).await { Ok(_) => StatusCode::CREATED, Err(_) => StatusCode::INTERNAL_SERVER_ERROR, } } #[tokio::main] async fn main() { // Fail at startup, not on the first request, if the runtime isn't configured. encore::initialize(); let app = Router::new().route("/users/:name", post(add_user)); let addr = std::env::var("ENCORE_LISTEN_ADDR").unwrap_or_else(|_| { let port = std::env::var("PORT").unwrap_or_else(|_| "8080".to_string()); format!("0.0.0.0:{port}") }); let listener = tokio::net::TcpListener::bind(&addr).await.unwrap(); axum::serve(listener, app) .with_graceful_shutdown(encore::shutdown::signal()) .await .unwrap(); std::process::exit(encore::shutdown::drained().await.code()); } ``` The last lines hand shutdown to Encore: the server stops on SIGTERM or SIGINT and drains its requests, then Encore drains Pub/Sub handlers and flushes telemetry. See [Graceful shutdown](https://encore.dev/docs/rs/develop/shutdown). ## App layout An Encore app is a directory with an `encore.app` file. Each service is a directory below it with an `encore.service.json` file, and for Rust, a Cargo package: ``` /my-app ├── encore.app └── users // the users service ├── encore.service.json ├── Cargo.toml ├── migrations │ └── 1_create_users.up.sql └── src └── main.rs ``` `encore app new my-app --template rust` creates an app with one Rust service at its root, and `encore app init` makes an existing directory an app. The service file names the service and sets `"lang": "rust"`: ```json -- users/encore.service.json -- { "name": "users", "lang": "rust", "api": [ { "type": "http", "prefix": "/", "name": "users", "expose": true } ] } ``` The `api` entry tells Encore that the service serves HTTP itself under `/`, and `expose` routes requests from the API gateway to it. See [Services](https://encore.dev/docs/rs/primitives/services) for every field, and for apps with several services or a Cargo workspace. The package depends on `encore-dev`, and on Tokio and your web framework: ```toml -- users/Cargo.toml -- [package] name = "users" version = "0.1.0" edition = "2024" [dependencies] encore-dev = { path = "/path/to/encore-dev" } axum = "0.7" serde = { version = "1", features = ["derive"] } tokio = { version = "1", features = ["rt-multi-thread", "macros", "net"] } ``` > **Important** > > During early access, `encore-dev` is not published on crates.io. Add it as a `path` or `git` dependency pointing at the SDK source you were given. Keep the library name `encore`: the SDK's macros, such as `#[encore::test]`, refer to the crate by that name. ### Cargo features The crate compiles only the infrastructure providers you need, since their clients account for most of its build time. None are on by default: | Feature | What it adds | |---|---| | `local` | The providers `encore run` and `encore test` use: the local object storage emulator and Pub/Sub broker. | | `aws` | S3 buckets and SNS/SQS Pub/Sub. | | `gcp` | Cloud Storage buckets and Google Cloud Pub/Sub. | | `protobuf` | Pub/Sub topics whose messages are Protocol Buffers. | You don't enable the provider features yourself. `encore run` and `encore test` build with `local`, and `encore build` with the clouds it builds for, using the name your `Cargo.toml` gives the dependency. A service with a [custom run or test command](https://encore.dev/docs/rs/primitives/services#custom-commands) has to pass `--features encore-dev/local` itself. Postgres, Redis, secrets and config need no feature. ## The runtime The SDK starts its runtime on first use, from the configuration Encore passes to the process. If that fails, for example because the binary was started with `cargo run` instead of `encore run`, the process prints `encore: failed to initialize the SDK runtime` and exits. Three functions control it: - `encore::initialize()` starts the runtime right away, so a misconfigured service fails at startup. It also starts the Pub/Sub subscriptions declared with `#[encore::pubsub::register]`. Call it at the top of `main`. - `encore::run_blocking()` starts the registered subscriptions, blocks until a shutdown signal, drains, and exits the process. It is for a service that serves no HTTP of its own, such as a Pub/Sub worker. Call it from a synchronous `main`, not from inside a Tokio runtime. - `encore::tokio_handle()` returns a handle to the Tokio runtime the SDK's core runs its background work on, for spawning tasks onto it. The SDK's methods are ordinary `async` functions. Your application brings its own Tokio runtime, typically with `#[tokio::main]`, and awaits them there. A worker service with no HTTP server: ```rust use encore::pubsub::{Message, Subscription, Topic}; static ORDERS: Topic = Topic::new("order-placed"); #[encore::pubsub::register] static FULFIL: Subscription = Subscription::new(&ORDERS, "fulfil", fulfil); async fn fulfil(msg: Message) -> Result<(), encore::Error> { println!("fulfilling {}", msg.data); Ok(()) } fn main() { encore::run_blocking(); } ``` ## Running and testing - `encore run` builds each service with `cargo build`, runs it with `cargo run`, provisions the infrastructure it declares, and rebuilds and restarts a service when its code changes. - `encore test` runs `cargo test` in each service, with infrastructure of its own for every test. See [Testing](https://encore.dev/docs/rs/develop/testing). - `encore build` builds the services for deployment, with the cloud providers compiled in. See [Build Docker Images](https://encore.dev/docs/self-host/build) for self-hosting. See the [CLI reference](https://encore.dev/docs/cli/cli-reference) for every command. ## The Encore.rs docs **Primitives** - [Services](https://encore.dev/docs/rs/primitives/services): the service file, how Cargo packages map to services, and service URLs. - [Databases](https://encore.dev/docs/rs/primitives/databases): Postgres databases, migrations and queries. - [Object Storage](https://encore.dev/docs/rs/primitives/object-storage): buckets and objects. - [Pub/Sub](https://encore.dev/docs/rs/primitives/pubsub): topics, subscriptions and ordered delivery. - [Cron Jobs](https://encore.dev/docs/rs/primitives/cron-jobs): requests Encore makes of a service on a schedule. - [Redis](https://encore.dev/docs/rs/primitives/redis): Redis databases, transactions and scripts. - [Secrets](https://encore.dev/docs/rs/primitives/secrets): service, shared and global secrets. **Develop** - [Configuration](https://encore.dev/docs/rs/develop/config): typed config values in files beside your code. - [Metadata](https://encore.dev/docs/rs/develop/metadata): the app's environment, build and deployment. - [Testing](https://encore.dev/docs/rs/develop/testing): isolated infrastructure per test, and fixtures. - [Graceful shutdown](https://encore.dev/docs/rs/develop/shutdown): draining requests and messages on shutdown. **Observability** - [Logging](https://encore.dev/docs/rs/observability/logging): structured logs through the `log` crate. - [Metrics](https://encore.dev/docs/rs/observability/metrics): counters, gauges and histograms. - [Tracing](https://encore.dev/docs/rs/observability/tracing): your own spans in Encore's traces. --- Source: https://encore.dev/docs/rs/primitives/services Section: Building (Encore.rs) # Services > Cargo packages Encore builds, runs and provisions for A service in an Encore.rs app is a directory with an `encore.service.json` file and a Cargo package that builds the service's binary. The service's code is ordinary Rust: it serves HTTP with your own web framework, and declares and uses infrastructure through the SDK. ## Defining a service Put an `encore.service.json` beside the service's `Cargo.toml`: ```json -- users/encore.service.json -- { "name": "users", "lang": "rust", "description": "Accounts and sign-in.", "api": [ { "type": "http", "prefix": "/users", "expose": true } ] } ``` Every file in the directory, and in the directories below it, belongs to the service. Services can't be nested: a directory belongs to exactly one service. On disk, an app with two Rust services might look like this: ``` /my-app ├── encore.app ├── users // users service │ ├── encore.service.json │ ├── Cargo.toml │ ├── migrations │ └── src │ └── main.rs └── billing // billing service ├── encore.service.json ├── Cargo.toml └── src └── main.rs ``` See [App Structure](https://encore.dev/docs/app-structure) for how services fit together in an app. ## The service file | Field | Meaning | |---|---| | `name` | The service's name, unique in the app. Required. | | `lang` | `"rust"`. Required. | | `description` | What the service is for, in a sentence or two. Optional. | | `api` | What the service serves, and which gateways route to it. See [Serving HTTP](#serving-http). | | `run` | How `encore run` starts the service. Defaults to `{ "type": "rust" }`. | | `build` | How `encore build` builds it. Defaults to `{ "type": "rust" }`. | | `test` | How `encore test` tests it. Defaults to `{ "type": "rust" }`. | | `docker` | How `encore build docker` packages it: `base_image` and `entrypoint`. | | `workers` | Its background processes, which serve no requests. See [Workers](https://encore.dev/docs/rs/primitives/workers). | ### Serving HTTP An `http` entry in `api` says the service serves everything at or below a path prefix itself. Encore doesn't need to know the routes under it, so the routing stays in your web framework. | Key | Meaning | |---|---| | `prefix` | The path the service serves, starting with `/`. | | `name` | A name for the entry, an identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to the prefix's last segment, so set it when the prefix is `/`. | | `expose` | `true` to route requests from the default API gateway to the service, a list of gateway names to route from those, or `false` (the default) to keep it internal. | | `methods` | The HTTP methods routed, such as `["GET", "POST"]`. Defaults to all of them. | A service with no `api` entries serves no HTTP from the gateway. A service that runs only [workers](https://encore.dev/docs/rs/primitives/workers) sets its `run` to `"none"`, and has no main process at all. `api` also takes `openapi`, `graphql` and `grpc` entries, which describe what the service serves with a spec document. ## How Encore runs a Rust service With the default `rust` strategies, Encore runs Cargo in the service's directory: | Command | What Encore runs | |---|---| | `encore run` | `cargo build`, then `cargo run`, with the SDK's `local` feature. | | `encore test` | `cargo test`, with the SDK's `local` feature. | | `encore build` | `cargo build --release`, with the features of the clouds it builds for. | Encore enables the features through the name your `Cargo.toml` gives the `encore-dev` dependency, so the package must depend on it directly (or through `workspace = true`). See [Cargo features](https://encore.dev/docs/rs#cargo-features). With the `rust` strategies, a service with a main process must build exactly one binary: that binary is the service. A service with [workers](https://encore.dev/docs/rs/primitives/workers) and a main process also has one `[[bin]]` per worker that uses `entry`; the service's own binary is the one no worker names as its `entry`. A service with `run: "none"` has no service binary and needs only the binaries or commands used by its workers. Under `encore run`, the service gets its address in two environment variables: - `ENCORE_LISTEN_ADDR`, the `host:port` to listen on. - `PORT`, the same port, for libraries that read only that. ```rust fn listen_addr() -> String { std::env::var("ENCORE_LISTEN_ADDR").unwrap_or_else(|_| { let port = std::env::var("PORT").unwrap_or_else(|_| "8080".to_string()); format!("0.0.0.0:{port}") }) } ``` When a service's code changes, `encore run` rebuilds and restarts it. ### Custom commands A `custom` strategy runs a command of your choosing instead: ```json { "name": "worker", "lang": "rust", "run": { "type": "custom", "cmd": ["cargo", "run", "--release", "--features", "encore-dev/local"] } } ``` | Key | Applies to | Meaning | |---|---|---| | `cmd` | run, build, test | The command and its arguments. | | `build` | run | A command run to completion before `cmd` each time the service starts. | | `output` | build | The path of the built artifact, relative to the output directory. | Encore adds no features to a custom command. Pass `--features encore-dev/local` (with the name you depend on the SDK by) in `run` and `test` commands, or buckets and Pub/Sub won't work locally. A service with a custom `build` must also set `docker.entrypoint` to be packaged as an image. ## Cargo workspaces Each service is its own Cargo package, but the packages can share a Cargo workspace, for example with a `Cargo.toml` at the app root: ```toml -- Cargo.toml -- [workspace] members = ["users", "billing", "libs/*"] resolver = "2" [workspace.dependencies] encore-dev = { path = "/path/to/encore-dev" } ``` ```toml -- users/Cargo.toml -- [package] name = "users" version = "0.1.0" edition = "2024" [dependencies] encore-dev = { workspace = true } shared = { path = "../libs/shared" } ``` A library crate a service depends on by path, directly or through the workspace, is part of that service's build: changing it rebuilds and restarts the services that use it. ## Which service uses what Encore attributes everything to the service whose directory contains the code: - **Usage.** Each call on a resource, such as `DB.query(…)` or `UPLOADS.upload(…)`, is recorded for the service the calling file belongs to, and that service is granted access to the resource. A call outside every service directory is an error, because there is no service to grant it to. - **Handles.** Creating a handle, such as `DB.read()` or `UPLOADS.reference::()`, is the usage. Library code outside the services takes the handle as an argument and records nothing itself. See [Databases](https://encore.dev/docs/rs/primitives/databases#passing-the-database-to-other-code). - **Ownership.** A few kinds of resource belong to the service that declares them: a Pub/Sub subscription is the declaring service's, a cron job requests a path on the declaring service, and a `Secret` or `Config` holds a value for the declaring service only. Databases, buckets, topics and Redis databases belong to the app, and any service may use them. To use a resource that another service declares, declare a second `static` for it by name, such as `Database::named("users")`. This works across languages too. ## Calling other services `encore::services::base_url` returns the base URL the calling service reaches another service at, without a trailing slash. Use it with any HTTP client: ```rust async fn fetch_user(id: i64) -> Result { let users = encore::services::base_url("users"); reqwest::get(format!("{users}/users/{id}")).await?.text().await } ``` Looking up a service's URL makes the calling service depend on it, and Encore reads that dependency from the code: - The service name must be a string literal. - The function must be called directly, not stored or passed around. - The call must be inside a service. `base_url` panics if the runtime knows no address for the service. That happens for a name that isn't a service of the app, and under `encore test`, which starts no other services. ## Static rules Encore reads services and their resources from the source, so the declarations have a fixed shape: - A resource is declared as a module-level `static`, with a `const` constructor such as `Database::new("users")`. A declaration inside a function, behind a condition, or wrapped in another call is an error. - Resource names are string literals. Options are chained onto the constructor, each set once from a literal. - A resource is used by calling its methods on the `static`. Assigning it to a variable or passing it to a function of your own is an error: create a handle instead. The SDK functions that take a resource, such as `Subscription::new(&TOPIC, …)` and the [testing](https://encore.dev/docs/rs/develop/testing) helpers, are the exception. - Every use of a resource is inside a service. Encore reports these errors when it parses the app, on `encore run`, `encore test` or `encore build`. --- Source: https://encore.dev/docs/rs/primitives/databases Section: Building (Encore.rs) # Databases > Provisioning, migrating, querying Encore.rs supports **PostgreSQL** databases. You declare a database as a `static`, write its schema as SQL migration files, and query it with methods modelled on [tokio-postgres](https://docs.rs/tokio-postgres). Encore provisions the database, applies the migrations, and grants each service the access its code uses. ## Creating a database Declare a `Database` as a module-level `static` in a service, and point it at a directory of migrations: ```rust -- todo/src/db.rs -- use encore::storage::postgres::Database; pub static DB: Database = Database::new("todo").migrations("./migrations"); ``` ```sql -- todo/src/migrations/1_create_table.up.sql -- CREATE TABLE todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL, done BOOLEAN NOT NULL DEFAULT false ); ``` `encore run` creates the database on a local Postgres server and applies the migrations before the service starts. Nothing else needs setting up. The declaration has a fixed shape, because Encore reads it from the source: - It is a `static` at module level, initialized by `Database::new(…)`. - The name is a string literal: lowercase letters, digits and underscores, starting with a letter and ending with a letter or digit, at most 63 characters, and unique in the app. Names starting with `encore_` are reserved. - The only option is `.migrations("…")`, chained onto `new` and set at most once. Its path is a string literal, relative to the directory of the file that declares the database. The directory must exist. A database declared without `.migrations` has no schema of its own. ## Migrations A migration is a SQL file named `_.up.sql`: - `` is a number. Migrations are applied in order of it, lowest first. Leading zeros are allowed (`0001_create_table.up.sql`). - `` is anything that says what it does. - Files with any other name are ignored. A `_.down.sql` file is the matching down migration, run only when you migrate down by hand. ``` /todo ├── encore.service.json ├── Cargo.toml └── src ├── main.rs ├── db.rs └── migrations ├── 1_create_table.up.sql └── 2_add_created.up.sql ``` Encore records the version a database is at in its `schema_migrations` table, and applies only the migrations above it. Each migration file runs as one batch of statements. ### Changing the schema Change the schema by adding a migration with a higher version than any before it. Don't edit a migration that has been applied: Encore won't run it again. ```sql -- todo/src/migrations/2_add_created.up.sql -- ALTER TABLE todo_item ADD created TIMESTAMPTZ NOT NULL DEFAULT now(); ``` A migration added while `encore run` is running shows as pending. Apply it with: ```shell encore db migrate --database todo --up ``` When you deploy, migrations are applied before the new version of the service starts. Make sure the code that is running keeps working with the new schema while the deploy rolls out. ### Handling migration errors If a migration fails, its statements are rolled back (unless the file commits on its own) and the database is marked dirty at that version. Encore won't migrate a dirty database until you say where it stands. Fix the migration, set the recorded version back to the last migration that applied, and migrate again: ```shell encore db version --database todo encore db migrate --database todo --set 1 encore db migrate --database todo --up ``` `encore db reset --database todo` drops a local database and creates it again from its migrations, discarding its data. ## Querying The query methods have the names, parameters and meaning of tokio-postgres's `Client` methods. They return `Result<_, encore::Error>`. Parameters are written `$1`, `$2`, … in the SQL and passed as a slice of references: ```rust use encore::storage::postgres::Database; static DB: Database = Database::new("todo").migrations("./migrations"); async fn add(title: &str) -> Result { let row = DB .query_one( "INSERT INTO todo_item (title) VALUES ($1) RETURNING id", &[&title], ) .await?; Ok(row.get("id")) } async fn open_titles() -> Result, encore::Error> { let rows = DB .query("SELECT title FROM todo_item WHERE NOT done ORDER BY id", &[]) .await?; Ok(rows.iter().map(|row| row.get("title")).collect()) } async fn complete(id: i64) -> Result { let updated = DB .execute("UPDATE todo_item SET done = true WHERE id = $1", &[&id]) .await?; Ok(updated == 1) } ``` | Method | Returns | |---|---| | `query(sql, params)` | Every row, as a `Vec`. | | `query_one(sql, params)` | Exactly one `Row`, or an error. | | `query_opt(sql, params)` | `Option`: an error if there is more than one row. | | `query_raw(sql, params)` | A `RowStream`, with the parameters as any iterator. | | `query_typed(sql, params)` | Like `query`, with each parameter's Postgres type given, so the query needs no round trip to prepare. | | `query_typed_raw(sql, params)` | The streaming form of `query_typed`. | | `execute(sql, params)` | The number of rows the statement changed. | | `execute_raw(sql, params)` | Like `execute`, with the parameters as any iterator. | | `simple_query(sql)` | Runs one or more statements with the simple query protocol, returning every value as text. | | `batch_execute(sql)` | Runs one or more statements with the simple query protocol, stopping at the first error. | `simple_query` and `batch_execute` take no parameters: never build their SQL from untrusted input. A query is a `&str` or `String`, or a prepared `Statement` (see [Connections](#connections)). Each query is traced. ### Reading rows `Row::get` reads a column by name or zero-based position, converting it to the type you ask for. It panics if the column doesn't exist or can't be converted; `Row::try_get` returns an error instead. ```rust let row = DB.query_one("SELECT id, title, done FROM todo_item WHERE id = $1", &[&id]).await?; let title: String = row.get("title"); let done: bool = row.get(2); let note: Option = row.try_get("note")?; ``` A `NULL` reads as `None` into an `Option`. `Row` also has `columns()`, `len()` and `is_empty()`. ### Streaming rows `query_raw` returns the rows as a `RowStream` rather than collecting them. Read it with `next()`, or use it as a `futures` `Stream`: ```rust let mut rows = DB .query_raw("SELECT title FROM todo_item WHERE id = ANY($1)", [ids]) .await?; while let Some(row) = rows.next().await { let title: String = row?.get("title"); println!("{title}"); } ``` After the last row, `rows_affected()` returns the number of rows the statement affected. ### Types Parameters implement `ToSql` and columns are read through `FromSql`, both in `encore::storage::postgres::types`. They cover every type tokio-postgres converts: integers, floats, `bool`, `&str` and `String`, `&[u8]` and `Vec`, `Option` for `NULL`, and slices and `Vec`s of them as arrays. The SDK also enables tokio-postgres's support for `serde_json::Value`, `uuid::Uuid` (uuid 1), the `chrono` 0.4 date and time types, `cidr` 0.3 and `geo-types` 0.7. Add the crate to your `Cargo.toml` to use its types. To convert a type of your own, implement the traits from `encore::storage::postgres::types`: ```rust use encore::storage::postgres::types::{to_sql_checked, BytesMut, FromSql, IsNull, ToSql, Type}; #[derive(Debug)] struct Celsius(f64); impl ToSql for Celsius { fn to_sql( &self, ty: &Type, out: &mut BytesMut, ) -> Result> { self.0.to_sql(ty, out) } fn accepts(ty: &Type) -> bool { ::accepts(ty) } to_sql_checked!(); } impl<'a> FromSql<'a> for Celsius { fn from_sql(ty: &Type, raw: &'a [u8]) -> Result> { f64::from_sql(ty, raw).map(Celsius) } fn accepts(ty: &Type) -> bool { ::accepts(ty) } } ``` ## Transactions `transaction()` begins a transaction on a connection of its own from the pool. The `Transaction` has the same query methods, and rolls back when dropped unless you commit it, so an early return with `?` can't leave one half done: ```rust async fn move_items(from: i64, to: i64) -> Result<(), encore::Error> { let tx = DB.transaction().await?; tx.execute("UPDATE todo_item SET list_id = $2 WHERE list_id = $1", &[&from, &to]) .await?; tx.execute("DELETE FROM todo_list WHERE id = $1", &[&from]).await?; tx.commit().await } ``` `rollback()` rolls back explicitly. To set the transaction's isolation level, access mode or deferrability, use `build_transaction()`: ```rust use encore::storage::postgres::IsolationLevel; let tx = DB .build_transaction() .isolation_level(IsolationLevel::Serializable) .read_only(true) .start() .await?; ``` ## Connections The database is a pool of connections. What needs a single connection, prepared statements and `COPY`, is on a `Connection` from `acquire()`, or on a `Transaction`: ```rust let conn = DB.acquire().await?; let by_id = conn.prepare("SELECT title FROM todo_item WHERE id = $1").await?; for id in [1i64, 2, 3] { let row = conn.query_one(&by_id, &[&id]).await?; println!("{}", row.get::<_, String>("title")); } conn.close().await; ``` - `prepare(sql)` and `prepare_typed(sql, types)` return a `Statement`, which can only be used on the connection that prepared it, or in a transaction on that connection. - `copy_in(sql)` runs a `COPY … FROM STDIN` and returns a sink to write the data to; `copy_out(sql)` runs a `COPY … TO STDOUT` and returns a stream of it. - `transaction()` and `build_transaction()` begin a transaction on the connection, which stays acquired after it ends. The connection goes back to the pool when it is closed or dropped. ## Passing the database to other code A `Database` can't be passed around as a value: Encore reads which service uses it, and how, from the calls made on the `static`. To hand it to other code, such as a repository struct or a library, make a `DatabaseRef` with the privileges that code needs: | Method | Privileges recorded for the service | |---|---| | `DB.read()` | Read. May use a read replica. | | `DB.read_primary()` | Read, always on the primary. | | `DB.write()` | Read and write. | | `DB.admin()` | Read, write and admin. | | a query method on `DB` itself | Read and write. | A `DatabaseRef` is `Copy`, and the methods are `const`, so a handle can also be a `static`. Library code takes the `Queryable` trait, which a `DatabaseRef`, a `Transaction` and a `Connection` implement, so the same code runs inside a transaction: ```rust use encore::storage::postgres::{Database, DatabaseRef, Queryable}; static DB: Database = Database::new("todo").migrations("./migrations"); static REPORTS: DatabaseRef = DB.read(); async fn open_count(db: &impl Queryable) -> Result { let row = db.query_one("SELECT count(*) FROM todo_item WHERE NOT done", &[]).await?; Ok(row.get(0)) } async fn report() -> Result { open_count(&REPORTS).await } async fn report_in_transaction() -> Result { let tx = DB.transaction().await?; let open = open_count(&tx).await?; tx.commit().await?; Ok(open) } ``` `Database` doesn't implement `Queryable`, so `open_count(&DB)` doesn't compile. > **Note** > > The privileges are what Encore records for the service. At runtime every handle currently shares the database's one read-write connection pool: `read()` doesn't yet route to a read replica or open a read-only connection. ## Using another service's database By default a database is used by the service that declares it, but any service can use it. Declare a second `static` with `Database::named` and the database's name: ```rust -- report/src/main.rs -- use encore::storage::postgres::Database; // The todo service's database. static TODO_DB: Database = Database::named("todo"); async fn completed() -> Result { let row = TODO_DB .query_one("SELECT count(*) FROM todo_item WHERE done", &[]) .await?; Ok(row.get(0)) } ``` A named database takes no options: it is configured where it is declared. The name must belong to a database declared somewhere in the app, in any language. Encore grants the `report` service access from the calls it makes, as for the declaring service. ## Using an ORM or another client `connection_string()` returns a connection string for the database, for clients other than the SDK's, such as [sqlx](https://github.com/launchbadge/sqlx), [Diesel](https://diesel.rs) or [SeaORM](https://www.sea-ql.org/SeaORM/): ```rust use encore::storage::postgres::Database; use sqlx::postgres::PgPool; static DB: Database = Database::new("todo").migrations("./migrations"); async fn pool() -> Result> { Ok(PgPool::connect(&DB.connection_string()?).await?) } ``` Calling it records read and write access for the service. Under `encore test` it returns the connection string of the test's own database. Keep the schema in Encore's migration files. An ORM or migration tool that generates plain SQL migration files works: name them as [above](#migrations) and point `.migrations` at the directory. ## Errors The methods return `encore::Error`, which has a `code` and a `message`: | Situation | `code` | |---|---| | An error from the database or the connection, including constraint violations. | `ErrCode::Unavailable` | | `query_one` found no rows. | `ErrCode::NotFound` | | `query_one` or `query_opt` found more than one row. | `ErrCode::FailedPrecondition` | | `Row::try_get` couldn't read the column. | `ErrCode::Internal` | The Postgres error, such as a duplicate key, is in the message. The SDK doesn't yet expose its SQLSTATE code. ## Seeding data To have data in every environment, such as lookup tables, insert it in a migration. To have data only while developing, insert it on startup when the app runs locally. Make the insert idempotent, since it runs on every start: ```rust use encore::env::CloudProvider; async fn seed() -> Result<(), encore::Error> { if matches!(encore::app_meta().environment.cloud, CloudProvider::Local) { DB.batch_execute(include_str!("../dev_data.sql")).await?; } Ok(()) } ``` `encore exec` runs a command with the Encore runtime attached to the app's local infrastructure. A one-off seed program can be a Cargo example in the service's package, such as `examples/seed.rs`, which keeps the package at the one binary a service builds: ```shell encore exec cargo run --example seed ``` For tests, use [fixtures](https://encore.dev/docs/rs/develop/testing#fixtures): a saved state each test starts from. ## Connecting to databases The CLI connects to the app's local databases: | Command | What it does | |---|---| | `encore db shell --database todo` | Opens a `psql` shell. | | `encore db conn-uri --database todo` | Prints a connection URI. | | `encore db proxy` | Runs a local proxy, so any Postgres client can reach the databases at `postgresql://localhost:/`. | | `encore db query --database todo "SELECT …"` | Runs one query and prints the result. | | `encore db list` | Lists the databases the app declares. | | `encore db version --database todo` | Prints the schema version. | | `encore db migrate --database todo` | Migrates with `--up`, `--down`, `--to N` or `--set N`. | | `encore db reset --database todo` | Drops the database and creates it again from its migrations. `--all` resets every one. | See the [CLI reference](https://encore.dev/docs/cli/cli-reference) for the options. --- Source: https://encore.dev/docs/rs/primitives/object-storage Section: Building (Encore.rs) # Object Storage > Simple and scalable storage APIs for files and unstructured data Object Storage is a simple and scalable way to store files and unstructured data. The most common implementation is Amazon S3, and its semantics are supported by every major cloud provider. Encore.rs gives you a cloud-agnostic API for object storage. Encore provisions the buckets your app declares: Amazon S3 or Google Cloud Storage in the cloud, and a local emulator under `encore run`. You also get: * Tracing of every object storage operation * Local development without any setup * A bucket of its own for each test under `encore test` ## Creating a bucket The core of object storage is the **bucket**, a collection of objects (files) addressed by key. Declare a bucket as a module-level `static`: ```rust -- src/storage.rs -- use encore::objects::Bucket; pub static PROFILE_PICTURES: Bucket = Bucket::new("profile-pictures"); ``` Encore reads the declaration from your source code to provision the bucket, so: * It must be a module-level `static`, not a local variable or a value built inside a function. * The name must be a string literal: lowercase letters, digits and dashes, starting with a letter. Options follow the name, each a method of its own: ```rust static AVATARS: Bucket = Bucket::new("avatars").public().versioned(); ``` | Option | Effect | | --- | --- | | `.public()` | Objects are readable by anyone, at the URL [`public_url`](#public-buckets) gives. | | `.versioned()` | An object keeps its earlier versions, which the `_with` operations can name. | The options are part of the declaration: they take the bucket by value, so they can't be called on the `static` later. ## Uploading objects `upload` stores bytes under a key and returns the stored object's attributes: ```rust let attrs = PROFILE_PICTURES.upload("user-123.jpg", image_bytes).await?; println!("stored {} bytes", attrs.size); ``` `upload_with` takes `UploadOptions` as well, to set the content type or a precondition: ```rust use encore::objects::{UploadOptions, UploadPreconditions}; PROFILE_PICTURES .upload_with( "user-123.jpg", image_bytes, UploadOptions { content_type: Some("image/jpeg".into()), // Fail rather than overwrite an existing object. preconditions: Some(UploadPreconditions { not_exists: Some(true) }), }, ) .await?; ``` If the precondition doesn't hold, the upload fails with `ErrCode::FailedPrecondition`. ### Streaming uploads To store an object without buffering it in memory, upload from a reader or a stream: * `upload_stream(key, reader, options)` takes anything that is `tokio::io::AsyncRead + Unpin + Send + Sync + 'static`, such as a `tokio::fs::File`. * `upload_chunks(key, chunks, options)` takes a `Stream` of `Result` chunks, where a chunk is anything that converts into `Bytes` (`Bytes`, `Vec`, `String`, …). For example, to store the body of an axum request as it arrives: ```rust use axum::body::Body; use axum::extract::Path; use axum::http::StatusCode; use encore::objects::{UploadOptions, UploadPreconditions}; async fn upload(Path(name): Path, body: Body) -> StatusCode { let options = UploadOptions { // Fail rather than overwrite an existing object. preconditions: Some(UploadPreconditions { not_exists: Some(true) }), ..Default::default() }; let stored = PROFILE_PICTURES .upload_chunks(&name, body.into_data_stream(), options) .await; match stored { Ok(_) => StatusCode::CREATED, Err(_) => StatusCode::INTERNAL_SERVER_ERROR, } } ``` If the stream fails partway, the upload fails and nothing is stored. This handler uses the key from the path as it is. In a real service, authenticate the caller, check they may write that key (or generate the key on the server), and limit the body size before uploading. ## Downloading objects `download` returns an object's full contents as a `Vec`: ```rust let data = PROFILE_PICTURES.download("user-123.jpg").await?; ``` `download_with` takes `DownloadOptions`: a `version` of a versioned bucket's object, or a `ByteRange` to read only part of the object: ```rust use encore::objects::{ByteRange, DownloadOptions}; // The 256 bytes starting at offset 1024. let part = PROFILE_PICTURES .download_with( "big.bin", DownloadOptions { range: Some(ByteRange { offset: 1024, length: Some(256) }), ..Default::default() }, ) .await?; ``` A `ByteRange` with `length: None` reads to the end of the object. A zero-length range is an `InvalidArgument` error. ### Streaming downloads `download_stream(key, options)` returns a `Download`, which hands you the object as it arrives. Take it: * chunk by chunk, with `download.next().await`, which returns `Option>`; * as a `futures_core::Stream` of the same items; * all at once, with `download.bytes().await`; * as a `tokio::io::AsyncRead`, with `download.into_async_read()`, for whatever takes a reader. ```rust use encore::objects::DownloadOptions; let mut download = PROFILE_PICTURES .download_stream("big.bin", DownloadOptions::default()) .await?; while let Some(chunk) = download.next().await { hasher.update(&chunk?); } // Or straight into a file. let mut reader = PROFILE_PICTURES .download_stream("big.bin", DownloadOptions::default()) .await? .into_async_read(); tokio::io::copy(&mut reader, &mut file).await?; ``` Since a `Download` is a `Stream`, it can be the body of an axum response: ```rust use axum::body::Body; let download = PROFILE_PICTURES .download_stream(&name, DownloadOptions::default()) .await?; let body = Body::from_stream(download); ``` ## Object attributes `attrs` returns an object's `ObjectAttrs`: its `name`, `size`, `content_type`, `etag`, and `version` (in a versioned bucket). `exists` says whether there is an object at the key: ```rust use encore::ErrCode; match PROFILE_PICTURES.attrs("user-123.jpg").await { Ok(attrs) => println!("{} bytes, {:?}", attrs.size, attrs.content_type), Err(err) if err.code == ErrCode::NotFound => println!("no such picture"), Err(err) => return Err(err), } if PROFILE_PICTURES.exists("user-123.jpg").await? { // … } ``` `attrs_with` and `exists_with` take `AttrsOptions` and `ExistsOptions`, whose `version` names a specific version. ## Listing objects `list` takes `ListOptions` — a key `prefix` and a `limit`, both optional — and returns a `ListIterator` of `ListEntry`s, each with the object's `name`, `size` and `etag`: ```rust use encore::objects::ListOptions; let mut entries = PROFILE_PICTURES .list(ListOptions { prefix: Some("user-".into()), ..Default::default() }) .await?; while let Some(entry) = entries.next().await { println!("{}", entry?.name); } ``` A `ListIterator` is also a `Stream`, so it works with `StreamExt` combinators. ## Removing objects `remove` deletes an object. `remove_with` takes `RemoveOptions`, whose `version` deletes a specific version: ```rust match PROFILE_PICTURES.remove("user-123.jpg").await { Ok(()) => {} Err(err) if err.code == ErrCode::NotFound => {} Err(err) => return Err(err), } ``` ## Versioned buckets In a bucket declared `.versioned()`, overwriting or removing an object keeps its earlier versions. `ObjectAttrs::version` identifies the version an upload stored, and every operation that reads or removes an object has a `_with` form whose options take a `version`: ```rust use encore::objects::{Bucket, DownloadOptions}; static DOCUMENTS: Bucket = Bucket::new("documents").versioned(); let first = DOCUMENTS.upload("contract.pdf", draft).await?; DOCUMENTS.upload("contract.pdf", signed).await?; // The draft is still there. let draft = DOCUMENTS .download_with( "contract.pdf", DownloadOptions { version: first.version.clone(), ..Default::default() }, ) .await?; ``` ## Public buckets A bucket declared `.public()` serves its objects to anyone over HTTP(S), without authentication. This suits static assets like images and videos. ```rust static PUBLIC_ASSETS: Bucket = Bucket::new("public-assets").public(); let url = PUBLIC_ASSETS.public_url("images/logo.png")?; ``` `public_url` is not `async`: it builds the URL without a request. On a bucket that isn't public it fails with `ErrCode::FailedPrecondition`. When you deploy with the Encore platform, Encore makes the bucket publicly readable and [configures a CDN](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) in front of it. When self-hosting, see how to configure public buckets in the [infrastructure configuration docs](https://encore.dev/docs/self-host/configure-infra). ## Signed URLs A signed URL lets someone without credentials upload or download one object directly, for a limited time. Anyone holding the URL can use it. ```rust use std::time::Duration; use encore::objects::SignedUrlOptions; // Valid for an hour. let upload = PROFILE_PICTURES.signed_upload_url("user-123.jpg").await?; // Valid for five minutes. let download = PROFILE_PICTURES .signed_download_url_with( "user-123.jpg", SignedUrlOptions { ttl: Some(Duration::from_secs(5 * 60)), }, ) .await?; println!("{} {}", upload.url, download.url); ``` A URL is valid for one hour unless `ttl` says otherwise, and for at most seven days. The client uploads with a `PUT` of the object's contents: ```bash curl -X PUT --data-binary @dog-wizard.jpeg "https://storage.googleapis.com/profile-pictures/user-123.jpg?x-goog-signature=b7a1<...>" ``` Signed URLs keep large or long-running transfers out of your service: the content flows between the client and the bucket, and only keys and metadata go through your code. For content that isn't private, prefer a [public bucket](#public-buckets): its URLs are served through a CDN. ## Errors Every operation returns `Result<_, encore::Error>`. Match on its `code` (an `encore::ErrCode`) to tell failures apart: | Code | When | | --- | --- | | `NotFound` | There is no object at the key (or no such version). | | `FailedPrecondition` | An upload precondition didn't hold, or `public_url` was called on a private bucket. | | `InvalidArgument` | The arguments were wrong: an empty byte range, a malformed version. | | `Unavailable` | The storage backend couldn't be reached or failed. | | `Internal` | Something else went wrong. | ## Passing a bucket to other code Encore reads which operations each service calls on a bucket — `upload`, `download`, `list`, … — and grants the service exactly those permissions. To make that possible, a `Bucket` can only be used by calling its methods on the `static` directly. It can't be stored, returned, or passed to a function. To hand a bucket to other code, make a `BucketRef` with `reference`, naming the permissions it needs: ```rust use encore::objects::{perms, Bucket, Uploader}; static UPLOADS: Bucket = Bucket::new("uploads"); // Library code asks for the permissions it needs, as traits. async fn save(bucket: &impl Uploader, key: &str) -> Result<(), encore::Error> { bucket.upload(key, b"hello".to_vec()).await?; Ok(()) } async fn greet() -> Result<(), encore::Error> { save(&UPLOADS.reference::(), "greeting.txt").await } ``` The service is granted every permission the reference names, whether or not the code it's passed to uses them. The permissions must be written as `reference`'s type argument: they're never inferred, not even from the type the reference is assigned to. The permissions are the types in `encore::objects::perms`. Each has a trait of the same name in `encore::objects` with the methods it grants: | Permission | Grants | | --- | --- | | `perms::Uploader` | `upload`, `upload_with`, `upload_stream`, `upload_chunks` | | `perms::Downloader` | `download`, `download_with`, `download_stream` | | `perms::Attrser` | `attrs`, `attrs_with`, `exists`, `exists_with` | | `perms::Lister` | `list` | | `perms::Remover` | `remove`, `remove_with` | | `perms::SignedUploader` | `signed_upload_url`, `signed_upload_url_with` | | `perms::SignedDownloader` | `signed_download_url`, `signed_download_url_with` | | `perms::PublicUrler` | `public_url` | | `perms::ReadWriter` | Every permission above except `PublicUrler` | Ask for several with a tuple. A `BucketRef` is `Copy`, and `reference` is `const`, so a reference can also initialize a `static`: ```rust use encore::objects::{perms, BucketRef, Lister, Uploader}; static AVATARS: BucketRef<(perms::Uploader, perms::Lister)> = UPLOADS.reference::<(perms::Uploader, perms::Lister)>(); async fn sync(bucket: &(impl Uploader + Lister)) -> Result<(), encore::Error> { let mut entries = bucket.list(Default::default()).await?; // … Ok(()) } ``` A `BucketRef` has the bucket's methods for the permissions it grants, so calling them doesn't need the traits in scope. Library code that takes the traits instead, as `save` and `sync` do above, works with any reference that grants them, and can be tested against a fake that implements them. ## Using a bucket from another service A bucket can be used from any service. In a service other than the one that declares it, bind it by name with `Bucket::named`: ```rust use encore::objects::Bucket; static UPLOADS: Bucket = Bucket::named("uploads"); ``` The name must be a string literal. A named bucket takes no options: it's configured where it's declared, which may be in a service written in another language. ## Testing Under `encore test`, each test gets a bucket of its own, empty unless a fixture fills it: what one test uploads, no other test sees. See [Testing](https://encore.dev/docs/rs/develop/testing). --- Source: https://encore.dev/docs/rs/primitives/pubsub Section: Building (Encore.rs) # Pub/Sub > Decoupling services and building asynchronous systems Publishers & Subscribers (Pub/Sub) let you build systems that communicate by broadcasting events asynchronously. This is a great way to decouple services for better reliability and responsiveness. Encore.rs lets you declare topics and subscriptions in code. Encore provisions them for each environment: a managed Pub/Sub service in the cloud, and a local broker under `encore run`, with no setup. Here is the whole flow: a topic, a subscription that handles its messages, and a publish. ```rust use encore::pubsub::{Message, Subscription, Topic}; use serde::{Deserialize, Serialize}; #[derive(Clone, Serialize, Deserialize)] pub struct SignupEvent { pub user_id: String, } pub static SIGNUPS: Topic = Topic::new("signups"); #[encore::pubsub::register] static WELCOME: Subscription = Subscription::new(&SIGNUPS, "send-welcome", send_welcome); async fn send_welcome(msg: Message) -> Result<(), encore::Error> { println!("welcome, {}", msg.data.user_id); Ok(()) } async fn signed_up(user_id: String) -> Result<(), encore::Error> { SIGNUPS.publish(&SignupEvent { user_id }).await?; Ok(()) } ``` ## Creating a topic The core of Pub/Sub is the **topic**, a named channel on which you publish messages. A topic is typed by the message it carries, and declared as a module-level `static`: ```rust use encore::pubsub::Topic; use serde::{Deserialize, Serialize}; #[derive(Clone, Serialize, Deserialize)] pub struct SignupEvent { pub user_id: String, } pub static SIGNUPS: Topic = Topic::new("signups"); ``` Encore reads the declaration from your source code to provision the topic, so: * It must be a module-level `static`, not a local variable or a value built inside a function. * The name must be a string literal: lowercase letters, digits, dashes and periods (`user.signed-up`), starting with a letter. ### At-least-once delivery By default, a topic delivers each message to each subscription _at least once_: if Encore believes a message wasn't handled, it delivers it again. **So every subscription handler should be [idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning)**: handling a message twice should have the same effect as handling it once. Track what you've already done in a database, or make the action itself idempotent. ### Exactly-once delivery Declare a topic `.exactly_once()` to have the infrastructure minimize redelivery: ```rust static PAYMENTS: Topic = Topic::new("payments").exactly_once(); ``` A message may still be redelivered in rare cases, such as when the acknowledgement that it was handled is lost on the network, so handlers should still be idempotent where correctness matters. Exactly-once delivery doesn't deduplicate publishes either: publish the same message twice and it's delivered twice. Cloud providers limit the throughput of exactly-once topics: - AWS: 300 messages per second for the topic (see [AWS SQS Quotas](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html)). - GCP: at least 3,000 messages per second across all topics in the region (see [GCP Pub/Sub Quotas](https://cloud.google.com/pubsub/quotas#quotas)). `.exactly_once()` is part of the declaration: it takes the topic by value, so it can't be called on the `static` later. ### Ordered topics A topic delivers messages in no particular order, which lets them be handled in parallel. When the messages about one entity must be handled in the order they were published, declare an `OrderedTopic`. Its `publish` takes an ordering key: messages with the same key are delivered in the order they were published, and messages with different keys in no particular order. ```rust use encore::pubsub::OrderedTopic; #[derive(Serialize, Deserialize)] pub struct CartEvent { pub cart_id: u64, pub event: String, } pub static CART_EVENTS: OrderedTopic = OrderedTopic::new("cart-events"); async fn checkout(cart_id: u64) -> Result<(), encore::Error> { let key = cart_id.to_string(); // Delivered in this order, as they share a key. CART_EVENTS.publish(&CartEvent { cart_id, event: "checkout_started".into() }, &key).await?; CART_EVENTS.publish(&CartEvent { cart_id, event: "checkout_completed".into() }, &key).await?; Ok(()) } ``` A message with a given key isn't delivered until the one before it has been handled or given up on, so one message that keeps failing holds up the rest of its key ([head-of-line blocking](https://en.wikipedia.org/wiki/Head-of-line_blocking)). Log and alert on failures, and choose the subscription's retry policy with that in mind. Cloud providers limit the throughput of ordered topics: - AWS: 300 messages per second for the topic (see [AWS SQS Quotas](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html)). - GCP: 1 MBps for each ordering key (see [GCP Pub/Sub Resource Limits](https://cloud.google.com/pubsub/quotas#resource_limits)). An ordered topic can also be `.exactly_once()`. ### Message encodings A topic's second type argument says how its messages are encoded: | Encoding | Message type | | | --- | --- | --- | | `Json` (the default) | Any type that implements serde's `Serialize` and `Deserialize` | `Topic` | | `Raw` | `Vec`, passed through as it is | `Topic, Raw>` | | `Protobuf` | A [`prost`](https://docs.rs/prost) message type | `Topic` | ```rust use encore::pubsub::{Raw, Topic}; static BLOBS: Topic, Raw> = Topic::new("blobs"); ``` `Protobuf` needs the SDK's `protobuf` feature: ```toml -- Cargo.toml -- [dependencies] encore-dev = { path = "/path/to/encore-dev", features = ["protobuf"] } ``` The encoding is part of the topic's type, so publishers and subscribers can't disagree about it. ## Publishing messages Call `publish` on the topic. It returns the message's id once the message has been published: ```rust let id = SIGNUPS.publish(&SignupEvent { user_id: "ada".into() }).await?; ``` Once `publish` returns, every subscription to the topic will receive the message. Subscribers see the same id. To send attributes alongside a message, use `publish_with`: ```rust use encore::pubsub::PublishOptions; SIGNUPS .publish_with( &event, PublishOptions { attributes: [("source".to_string(), "import".to_string())].into(), }, ) .await?; ``` On an `OrderedTopic`, `publish` and `publish_with` take the ordering key after the message. Encore reads which services publish to which topics from the calls to `publish`, and grants each of them permission to. So a topic can only be used by calling its methods on the `static` directly: it can't be stored, returned, or passed to a function. If library code needs to publish, give it a function that does. ## Subscribing to messages A subscription delivers every message published to its topic to a handler. Declare it as a module-level `static` with `Subscription::new`, taking the topic, a subscription name, and the handler: ```rust use encore::pubsub::{Message, Subscription}; #[encore::pubsub::register] static WELCOME: Subscription = Subscription::new(&SIGNUPS, "send-welcome", send_welcome); async fn send_welcome(msg: Message) -> Result<(), encore::Error> { // Send the email… Ok(()) } ``` * The name must be a string literal, unique for the topic: lowercase letters, digits and dashes. * The handler is an `async fn` taking a `Message` and returning `Result<(), encore::Error>`. Its future must be `Send`. * The subscription runs in the service that declares it, which can be the topic's service or any other. Each subscription receives every message, independently of the topic's other subscriptions: if one handler is slow, its subscription builds up a backlog while the others keep up. ### The message A handler receives a `Message`, with these fields: | Field | Type | | | --- | --- | --- | | `id` | `String` | The message's id, unique within its topic. | | `data` | `T` | The message, decoded. | | `raw_data` | `Vec` | The message as it was sent, before decoding. | | `attributes` | `HashMap` | The attributes it was published with. | | `publish_time` | `SystemTime` | When it was published. | | `delivery_attempt` | `u32` | Which attempt at delivering it this is, from 1. | | `ordering_key` | `Option` | The key it was published with, on an `OrderedTopic`. | ### Starting subscriptions A `static` does nothing until something uses it, so something has to start a subscription. `#[encore::pubsub::register]` registers one to start with the app. Registered subscriptions start when your `main` calls `encore::initialize()`: ```rust #[tokio::main] async fn main() { // Connects to Encore's runtime and starts the registered subscriptions. encore::initialize(); // Serve HTTP with your framework of choice… } ``` A service that only handles messages, and serves no requests of its own, can call `encore::run_blocking()` from a plain (non-async) `main` instead. It starts the registered subscriptions and blocks until the process is told to shut down. ```rust fn main() { encore::run_blocking(); } ``` A subscription declared without the attribute is started by hand with `WELCOME.start()`. Without the attribute, the handler must be written as a closure that boxes the future: `|msg| Box::pin(send_welcome(msg))`. The attribute is what lets you give the plain name of an `async fn`. If a service declares a subscription and never starts it, Encore logs a warning shortly after the service starts. When the process is told to shut down, Encore stops taking new messages and waits for in-flight handlers to finish. See [Graceful shutdown](https://encore.dev/docs/rs/develop/shutdown). ### Subscription options Options follow the handler, each a method of its own: ```rust use std::time::Duration; #[encore::pubsub::register] static WELCOME: Subscription = Subscription::new(&SIGNUPS, "send-welcome", send_welcome) .ack_deadline(Duration::from_secs(60)) .max_retries(5) .max_concurrency(4); ``` | Option | Default | | | --- | --- | --- | | `.ack_deadline(Duration)` | 30 seconds | How long the handler has to finish before the message is delivered again. | | `.message_retention(Duration)` | 7 days | How long an undelivered message is kept. | | `.min_backoff(Duration)` | 10 seconds | The shortest wait before a failed message is delivered again. | | `.max_backoff(Duration)` | 10 minutes | The longest wait before a failed message is delivered again. | | `.max_retries(i32)` | 100 | How many times a failed message is retried before it's given up on. `-1` retries forever, `-2` never retries. | | `.max_concurrency(u32)` | Unlimited | How many messages are handled at once. | Encore reads the options from your source code to configure the infrastructure, so write them where the subscription is declared, with literal values: durations as `Duration::from_secs(…)` or `Duration::from_millis(…)`, and numbers as literals. Each option can be set once. ### Error handling If a handler returns an error, or a message can't be decoded as the topic's type, the message is delivered again later, with a backoff between `min_backoff` and `max_backoff`. After `max_retries` failed retries it's moved to the subscription's dead-letter queue, so the subscription can keep handling other messages until the bug is fixed. You can then release dead-lettered messages to be processed again. Locally, use the `encore pubsub` commands to see and manage what happened to each message: ```shell $ encore pubsub ls # topics, subscriptions and their queues $ encore pubsub queue user-added greet --state dead $ encore pubsub show # a message and its delivery attempts $ encore pubsub release user-added greet # deliver dead messages again ``` ## Using a topic from another service Any service can publish to a topic and subscribe to it. In a service other than the one that declares it, bind it by name with `Topic::named` (or `OrderedTopic::named`): ```rust use encore::pubsub::Topic; static SIGNUPS: Topic = Topic::named("signups"); ``` The name must be a string literal. A named topic takes no options: it's configured where it's declared, which may be a service in another language. Its message type and encoding are your claim: across languages Encore can't check them, so make sure they match what's published. ## Testing Under `encore test`, each test publishes to topics of its own, and no subscription runs: nothing handles what a test publishes. That lets you test publishers without the side effects of subscribers. To check what was published, `encore::testing::published` returns what the current test, and the code it called, published to a topic, in order. To test a handler, call it with what was published: ```rust #[cfg(test)] mod tests { use super::*; #[encore::test] async fn signing_up_sends_a_welcome() -> Result<(), encore::Error> { signed_up("ada".to_string()).await?; let published = encore::testing::published(&SIGNUPS)?; assert_eq!(published.len(), 1); assert_eq!(published[0].data.user_id, "ada"); // Run the subscription's handler on it. send_welcome(published[0].clone()).await?; Ok(()) } } ``` `Message` is `Clone` when `T` is. See [Testing](https://encore.dev/docs/rs/develop/testing) for more. ## Ensuring consistency between services Database writes and publishes aren't transactional: a service can write to its database and then fail to publish, or the other way around. If that matters, consider the [transactional outbox pattern](https://microservices.io/patterns/data/transactional-outbox.html): write the message to an outbox table in the same transaction as the change, and publish from the table. --- Source: https://encore.dev/docs/rs/primitives/cron-jobs Section: Building (Encore.rs) # Cron Jobs > Run recurring and scheduled tasks When you need to run periodic and recurring tasks, Encore.rs lets you declare cron jobs in code. When a job is due, Encore sends a `POST` request for the job's path to the service that declares it. Whatever the service serves at that path is the job. Encore handles the scheduling, monitoring and execution, so there's no scheduler for you to run. > **Note** > > Cron jobs are not scheduled by `encore run` or `encore test`: a job runs where the app is deployed. To try one locally, [run it by hand](#running-a-job-by-hand). ## Defining a cron job Declare a job as a module-level `static` with `Job::new`, giving its name, the path it requests, and its schedule: ```rust use encore::cron::{Job, Schedule}; /// Removes the sessions nobody has used in a month. static CLEANUP: Job = Job::new("cleanup", "/internal/cleanup", Schedule::Cron("0 4 * * *")).title("Nightly cleanup"); static REFRESH: Job = Job::new("refresh-rates", "/internal/rates", Schedule::Every("30m")); ``` `.title(…)` is optional: it's the name people see for the job, in place of its name. The name is a unique ID for the job. If you later move the declaration to another file, Encore uses the name to know it's the same job, not a new one. ## Serving the job A job's path is served by your web framework like any other route. Keep the route and the job in step by routing the job's own path, `JOB.path()`: ```rust use axum::http::StatusCode; use axum::routing::post; use axum::Router; async fn cleanup() -> StatusCode { // Remove the stale sessions… StatusCode::OK } let app = Router::new().route(CLEANUP.path(), post(cleanup)); ``` The request is a `POST` with no body. Make the handler [idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning): under certain network conditions, a job may be requested more than once for the same run. ## The rules Encore reads the declaration from your source code, so: * A job must be a module-level `static`, declared inside a service: its path is requested of the service that declares it. * The name and the path must be string literals, and the schedule must be written out in the declaration, as `Schedule::Cron("…")` or `Schedule::Every("…")` with a string literal. * The name is lowercase letters, digits and dashes, starting with a letter, and unique in the app. * The path starts with a `/`. * The only option is `.title(…)`, with a string literal. Encore reports a declaration that breaks these rules when it parses your app, for example on `encore run`. ## Schedules ### Every `Schedule::Every` runs a job at a fixed interval, written as a duration of hours and minutes: `"30m"`, `"6h"`, `"1h30m"`. The runs are laid out over the day from midnight UTC, the same every day: `"6h"` runs at 00:00, 06:00, 12:00 and 18:00. So that the interval between runs stays the same, including from the last run of one day to the first of the next, the interval: * is a whole number of minutes, at most 24 hours, and * **divides 24 hours evenly**: `"45m"` and `"8h"` do, `"50m"` and `"7h"` do not. Encore reports an interval that doesn't, and suggests the nearest one that does. ### Cron expressions For schedules an interval can't express, such as a specific day of the month or week, use `Schedule::Cron` with a [cron expression](https://en.wikipedia.org/wiki/Cron#Cron_expression). It has five fields — minute, hour, day of month, month and day of week — and all times are in UTC: ```rust // At 04:00 UTC on the 15th of each month. static ACCOUNTING_SYNC: Job = Job::new( "accounting-sync", "/internal/accounting-sync", Schedule::Cron("0 4 15 * *"), ); ``` Each field is `*` (any), a value, a range (`9-17`), or a comma-separated list of them (`0,30`), each optionally with a step (`*/15`, `0-30/10`). Months and weekdays can be written by name (`JAN`–`DEC`, `SUN`–`SAT`, in any case), and Sunday is `0`. Some examples: | Expression | Runs | | --- | --- | | `0 4 * * *` | 04:00 every day | | `*/15 * * * *` | Every 15 minutes | | `0 9 * * 1` | 09:00 on Mondays | | `0 9-17 * * MON-FRI` | On the hour, 09:00 to 17:00, on weekdays | ## Running a job by hand Nothing runs a job locally on its own. To run one, do what Encore does when it's due: send a `POST` for its path to the service, while `encore run` is running: ```bash curl -X POST localhost:/internal/cleanup ``` In tests, call the function that serves the path. ## Keep in mind - Cron jobs don't run during local development or under `encore test`. - A job's handler takes no input: the request has no body. --- Source: https://encore.dev/docs/rs/primitives/workers Section: Building (Encore.rs) # Workers > Background processes of a service A worker is a long-running process of a service that serves no requests. It runs next to the service's main process, from a binary of its own in the service's Cargo package, and does whatever that binary does: consume Pub/Sub messages, run a Temporal worker, poll a queue, reconcile state on a timer. A worker *is* its service at runtime. It has the service's secrets, configs, databases and permissions, and it calls other services as the service does. There is nothing to grant or share: code a worker uses is the service's code. ## Enabling workers Workers are experimental: they work, but how they are declared may still change. An app uses them only by enabling the `workers` feature in its `encore.app.json`: ```json -- encore.app.json -- { "features": ["workers"] } ``` Without it, Encore refuses to parse an app that declares workers, with an error saying what to add. ## Declaring workers Workers are declared in the service's `encore.service.json`, by name. A Rust worker's `entry` is a `[[bin]]` of the service's package: ```json -- orders/encore.service.json -- { "name": "orders", "lang": "rust", "api": [ { "type": "http", "prefix": "/", "name": "orders", "expose": true } ], "workers": { "fulfillment": { "entry": "fulfillment-worker" }, "reconciler": { "entry": "reconciler", "instances": "one", "shutdown_timeout": "2m" } } } ``` ```toml -- orders/Cargo.toml -- [package] name = "orders" # ... [[bin]] name = "orders" path = "src/main.rs" [[bin]] name = "fulfillment-worker" path = "src/bin/fulfillment_worker.rs" [[bin]] name = "reconciler" path = "src/bin/reconciler.rs" ``` The service's own binary is the one left that no worker names. | Field | Required | Description | |---|---|---| | `entry` | Yes, unless `run` is given | The `[[bin]]` the worker runs. | | `description` | No | A sentence or two about what the worker is for, shown in Encore's tools. | | `instances` | No | `"any"` (the default), or `"one"` for code that must never run twice at once, such as a leader loop. A deployment then runs a single copy of it, and stops the old one before starting a new one. This is best effort: take a lock where correctness depends on it. | | `shutdown_timeout` | No | How long the worker may take to stop once told to, as `"30s"` or `"10m"`. Defaults to the deployment's graceful shutdown. | | `run` | No | How `encore run` starts it, as a service's `run`, instead of from `entry`. | | `docker` | No | `{ "entrypoint": [...] }`: the command its process runs in a Docker image, instead of its binary. | A worker's name is lowercase letters, digits and underscores, unique within its service. Within the app, Encore calls it `.`, as in `orders.fulfillment`. ## Writing a worker A worker's binary starts Encore and runs until it is told to stop. One that only consumes Pub/Sub messages calls `encore::run_blocking()`, which starts the registered subscriptions, blocks until shutdown, drains and exits: ```rust -- orders/src/bin/fulfillment_worker.rs -- use orders::subscriptions as _; // registers the subscriptions this worker consumes fn main() { encore::run_blocking(); } ``` One with work of its own runs it until shutdown begins, then lets it finish. See [Graceful shutdown](https://encore.dev/docs/rs/develop/shutdown) for how: ```rust -- orders/src/bin/reconciler.rs -- #[tokio::main] async fn main() { encore::initialize(); // At startup, before the signal can arrive. let stop = encore::shutdown::signal(); tokio::pin!(stop); loop { tokio::select! { _ = &mut stop => break, _ = tokio::time::sleep(std::time::Duration::from_secs(60)) => { // compare and fix state, using Encore resources } } } std::process::exit(encore::shutdown::drained().await.code()); } ``` `encore::env::app_meta().deploy.worker` is the name of the worker a process runs, and `None` in the service's main process. ## Consuming Pub/Sub messages A subscription names the worker that consumes it with `.worker(...)`: ```rust -- orders/src/subscriptions.rs -- use encore::pubsub::{Message, Subscription}; use crate::topics::{Order, ORDERS}; #[encore::pubsub::register] static FULFIL_ORDERS: Subscription = Subscription::new(&ORDERS, "fulfil-orders", fulfil).worker("fulfillment"); async fn fulfil(msg: Message) -> Result<(), encore::Error> { Ok(()) } ``` Only that worker's process takes the subscription's messages. The service's main process, and its other workers, take none, even when they link the module that declares it. Without `.worker(...)`, the service's main process consumes the subscription, as it always has. Encore reads the worker's name from the source, so write it as a string, and it must name a worker the service declares. ## A service with only workers A service that serves nothing has no main process: set its `run` to `"none"`. ```json -- indexer/encore.service.json -- { "name": "indexer", "lang": "rust", "run": "none", "workers": { "index": { "entry": "index-worker" } } } ``` Such a service can't declare an `api`, a frontend or a gateway, or cron jobs, which call a path of their service. Each subscription it declares needs a `.worker(...)`, since no main process would consume it. ## Cron jobs A cron job calls an HTTP path of its service, which only the main process serves, so it can't target a worker. Declare it in the service, and have its endpoint publish a message a worker consumes if the work belongs in the worker. ## Running workers locally Under `encore run`, a service's workers start when the service does, and keep running while anything keeps the service up. Unlike the main process, a worker is never stopped for being idle, since no request would start it again. A message waiting on a subscription a worker consumes starts the worker too. Each worker is built with `cargo build --bin ` and shows up in `encore run` and the dashboard as `.`. Its log records are its service's, with an `encore.worker.name` attribute naming the worker. It counts as running once it has stayed up for 250 milliseconds; one that exits sooner failed to start, and its exit and last lines are shown. A worker that crashes stays down until the next rebuild, as a service does. When the service's source changes, its workers are rebuilt with it: each is stopped, then started again, so a worker never runs twice. ## Deploying workers `encore build docker` packages a service's workers in its image, each binary at `bin/` beside the service's, and the image runs them beside the service. A deployment can also run them on their own, in containers that run only workers. Its infra config's `hosted_workers` lists the workers a container runs, by `.`, and `encore infra config generate` lists each worker where its service runs. A worker that runs with its service scales with it. On compute that throttles CPU between requests, as Cloud Run does by default, a worker sharing a container with its service needs CPU allocated all the time. --- Source: https://encore.dev/docs/rs/primitives/redis Section: Building (Encore.rs) # Redis > Fast in-memory data for caching, counters, rate limiting and more [Redis](https://redis.io/) is an in-memory data store, commonly used as a cache, for counters and rate limits, for short-lived state like sessions and locks, and for lightweight queues. Encore.rs lets you declare Redis databases in code. Encore provisions them for each environment: a managed Redis in the cloud, and a local one under `encore run`, with no setup. Under `encore test`, each test gets a database of its own. > **Note** > > Encore.go and Encore.ts expose Redis as *caching*: you declare a cache cluster, and typed keyspaces within it. Encore.rs has neither. A Redis database is a resource of its own, its keys are whatever your code passes, and its commands are Redis's own, grouped by data type: > > | Encore.go / Encore.ts | Encore.rs | > | --- | --- | > | Cache cluster | `Database::new("name")` | > | Cluster eviction policy | `.eviction_policy(…)` on the database | > | Keyspace with a key pattern | Keys you build yourself, such as `format!("user:{id}")` | > | Keyspace value type | The type you read a reply as, or `Json` for structured values | > | Default expiry | `set_ex`, `expire`, or `SetOptions` on each write | > > A database can also be declared `.persistent()`, for data that must not be lost or evicted. ## Creating a database Declare a database as a module-level `static`: ```rust -- src/sessions.rs -- use encore::storage::redis::Database; static SESSIONS: Database = Database::new("sessions"); async fn record_visit(user: &str) -> Result { let key = format!("visits:{user}"); let visits = SESSIONS.string().incr(&key, 1).await?; SESSIONS.generic().expire(&key, 24 * 60 * 60).await?; Ok(visits) } ``` Encore reads the declaration from your source code to provision the database, so: * It must be a module-level `static`, not a local variable or a value built inside a function. * The name must be a string literal: lowercase letters, digits and dashes, starting with a letter. The connection pool is built the first time the database is used, and reused after that. ### Options Options follow the name, each a method of its own: ```rust use encore::storage::redis::{Database, EvictionPolicy}; // A cache: what it holds may be evicted, least recently used first. static SESSIONS: Database = Database::new("sessions"); // Data that must survive a restart, and is never evicted. static LEDGER: Database = Database::new("ledger").persistent(); // A cache that evicts the least frequently used keys first. static THUMBNAILS: Database = Database::new("thumbnails").eviction_policy(EvictionPolicy::AllKeysLfu); ``` | Option | Effect | | --- | --- | | `.persistent()` | The data survives a restart and is never evicted. Without it, a database is a cache: what it holds may be lost or evicted. | | `.eviction_policy(policy)` | What to evict when the database runs out of memory. `EvictionPolicy::AllKeysLru` unless set. A persistent database evicts nothing and takes no policy. | The policies are `NoEviction`, `AllKeysLru`, `AllKeysLfu`, `AllKeysRandom`, `VolatileLru`, `VolatileLfu`, `VolatileTtl` and `VolatileRandom`, with the same meaning as Redis's [`maxmemory-policy`](https://redis.io/docs/latest/develop/reference/eviction/) values. The `Volatile` policies only evict keys that have an expiry. The options are part of the declaration: they take the database by value, so they can't be called on the `static` later. ## Running commands Commands are grouped by the data type they work on, each group reached through a method of the database: | Group | Commands on | | --- | --- | | `generic()` | Any key: `exists`, `del`, `expire`, `ttl`, `rename`, `scan`, … | | `string()` | Plain values: `get`, `set`, `set_ex`, `set_nx`, `mget`, `mset`, `incr`, `decr`, `append`, … | | `bitmap()` | The bits of a string: `getbit`, `setbit`, `bitcount`, `bit_and`, … | | `hash()` | Hashes: `hget`, `hset`, `hset_multiple`, `hgetall`, `hdel`, `hincr`, `hscan`, … | | `list()` | Lists: `lpush`, `rpush`, `lpop`, `rpop`, `lrange`, `lmove`, … | | `set()` | Sets: `sadd`, `srem`, `smembers`, `sismember`, `sinter`, `sscan`, … | | `sorted_set()` | Sorted sets: `zadd`, `zrange`, `zrange_withscores`, `zscore`, `zincr`, `zscan`, … | | `geo()` | Geospatial indexes: `geo_add`, `geo_pos`, `geo_dist`, `geo_radius`, … | | `stream()` | Streams: `xadd`, `xrange`, `xread`, `xread_group`, `xack`, … | | `pubsub()` | Channels: `publish`, `subscribe`, `psubscribe` | Within a group, commands are named as in the [`redis`](https://docs.rs/redis/latest/redis/) crate and take what its conversion traits accept: integers, floats, `bool`, strings, byte slices, and `Vec`s, slices and tuples of them. A slice or a `Vec` of keys is how you pass several keys to commands like `del` and `mget`. ```rust SESSIONS.string().set("name:1", "Ada").await?; SESSIONS.string().set_ex("token:abc", "user-1", 3600).await?; SESSIONS.hash().hset_multiple("user:1", &[("name", "Ada"), ("plan", "pro")]).await?; SESSIONS.list().rpush("queue", &["job-1", "job-2"]).await?; SESSIONS.sorted_set().zadd("scores", "ada", 42.0).await?; SESSIONS.generic().del(&["name:1", "token:abc"]).await?; ``` ### Reading replies Commands that reply with a fixed type, like `exists` (`bool`) or `llen` (`u64`), return it. Where a reply could be read several ways, as with `get`, the type is yours to choose: ```rust use std::collections::HashMap; let name: Option = SESSIONS.string().get("name:1").await?; let visits: u64 = SESSIONS.string().get("visits:ada").await?; let user: HashMap = SESSIONS.hash().hgetall("user:1").await?; let top: Vec<(String, f64)> = SESSIONS.sorted_set().zrevrange_withscores("scores", 0, 9).await?; ``` Read a value that may be missing as an `Option`, which is `None` for a missing key. A reply that doesn't convert to the type you asked for fails with `ErrCode::Internal`. `incr` takes an `i64` or an `f64`, and replies with the same type, so `incr("hits", 1)` returns an `i64` and `incr("score", 0.5)` an `f64`. ### Expiry and conditional writes Set an expiry with `set_ex` or `generic().expire`, or use `set_options` for `SET`'s full set of options, re-exported from the `redis` crate: ```rust use encore::storage::redis::{ExistenceCheck, SetExpiry, SetOptions}; // Take a lock for 30 seconds, unless someone holds it. let reply: Option = LOCKS .string() .set_options( format!("lock:{job}"), token, SetOptions::default() .conditional_set(ExistenceCheck::NX) .with_expiration(SetExpiry::EX(30)), ) .await?; let acquired = reply.is_some(); ``` ### Structured values Wrap a value in `Json` to store it as JSON, and read it back the same way. It works with any type that implements serde's `Serialize` and `Deserialize`: ```rust use encore::storage::redis::Json; use serde::{Deserialize, Serialize}; #[derive(Serialize, Deserialize)] struct Profile { name: String, plan: String, } SESSIONS.string().set("profile:1", Json(&profile)).await?; let Json(profile): Json> = SESSIONS.string().get("profile:1").await?; ``` A missing value reads as JSON's `null`, so read one that may be missing as `Json>`. To store a type of your own in another way, implement the `ToRedisArgs` and `FromRedisValue` traits from `encore::storage::redis`. ## Call commands in one expression Encore reads which commands each service runs on a database and grants the service the access it needs: reading or writing, per group. For that, a command has to be called in the same expression that names its group. A group that's stored or passed on says nothing about what it will be used for, and Encore reports it as an error: ```rust // Good: Encore sees `get` on the string group. let name: Option = SESSIONS.string().get("name:1").await?; // Error: Encore can't tell which command runs on `strings`. let strings = SESSIONS.string(); let name: Option = strings.get("name:1").await?; ``` Likewise, a `Database` can only be used by calling its methods on the `static`: it can't be stored, returned, or passed to a function. To hand a database to other code, take a handle. ## Handles A handle is a `Copy` value with the access its code needs, which you can pass around freely: | Method | Returns | Can | | --- | --- | --- | | `DB.write()` | `DatabaseRef` | Read and write: every command, pipelines, transactions, scripts | | `DB.read()` | `ReadOnlyDatabaseRef` | Run the commands that only read, subscribe, and run read-only scripts | | `DB.read_primary()` | `ReadOnlyDatabaseRef` | The same, always on the primary server | ```rust use encore::storage::redis::{Database, ReadOnlyDatabaseRef}; static SESSIONS: Database = Database::new("sessions"); // Library code takes a handle, and can only do what the handle allows. async fn name_of(db: ReadOnlyDatabaseRef, user: &str) -> Result, encore::Error> { db.string().get(format!("name:{user}")).await } async fn greet(user: &str) -> Result, encore::Error> { name_of(SESSIONS.read(), user).await } ``` The service is granted the handle's access to every group, whatever the code it's passed to does. The methods that make handles are `const`, so a handle can also initialize a `static`. > **Note** > > `read()` may in future connect to a read replica. Today every handle uses one connection pool, on the primary. ## Pipelines A pipeline sends several commands in one round trip, and returns their replies in order, as a tuple or a `Vec`. It's not a transaction: other clients' commands may run in between, and one command failing doesn't stop the rest. ```rust let mut pipe = SESSIONS.pipeline(); pipe.string().get("name:1"); pipe.string().get("name:2"); let (first, second): (Option, Option) = pipe.exec().await?; ``` Queueing a command returns the queue, so commands can also be chained: `pipe.string().get("a").get("b");`. ## Transactions A transaction (`MULTI`/`EXEC`) runs its commands together, with no other client's in between. Queue the commands, then `exec` it: ```rust let mut tx = LEDGER.transaction(); tx.string().set("account:ada", 100); tx.hash().hset("owners", "ada", "Ada Lovelace"); tx.string().get("account:ada"); let ((), added, balance): ((), u64, i64) = tx.exec().await?; ``` Nothing is sent until `exec`, so a transaction is never left open across an `.await`. To abandon one, drop it, or call `discard()`. ### Watching keys For a transaction that depends on values you read first, use `watch`. It runs your closure on a connection of its own: the closure reads what it needs through the connection and queues its changes on the transaction. The changes are applied only if none of the watched keys changed in between. If one did, the closure runs again. ```rust let left: i64 = LEDGER .watch("account:ada", async |conn, tx| { let balance: i64 = conn.string().get("account:ada").await?; tx.string().set("account:ada", balance - 10); Ok(balance - 10) }) .await?; ``` `watch` takes one key or several, and gives up with `ErrCode::Aborted` if the keys keep changing. The closure is an async closure, which needs Rust 1.85 or later. For finer control, `acquire()` takes a `Connection` out of the pool, with its own `watch`, `unwatch`, `transaction` and `pipeline`. Its transaction's `exec` returns `None` if a watched key changed. The connection goes back to the pool when dropped. ## Scripts A Lua script runs on the server as one step. Declare it as a `static` with `Script::new`, then call it with its keys and arguments: ```rust use encore::storage::redis::{Database, Script}; static LOCKS: Database = Database::new("locks"); static RELEASE: Script = Script::new( r#" if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) end return 0 "#, ); async fn release(job: &str, token: &str) -> Result { LOCKS .script(&RELEASE) .key(format!("lock:{job}")) .arg(token) .invoke() .await } ``` Each `.key(…)` is the next of the script's `KEYS`, and each `.arg(…)` the next of its `ARGV`. A script may only touch the keys it's passed through `KEYS` (see [Keys](#keys)). A script declared with `Script::read_only` can only read, which the server enforces. It's the only kind a `ReadOnlyDatabaseRef` can run. To run a script inside a pipeline or a transaction, queue it: `tx.script(&RELEASE).key(…).arg(…).queue();`. ## Scanning The scan methods iterate over a database's keys, or over the elements of one key, a batch per round trip. Each returns a `Scan`, whose `next` returns the next element: ```rust use encore::storage::redis::ScanOptions; let mut keys = SESSIONS .generic() .scan::(ScanOptions::default().pattern("user:*")); while let Some(key) = keys.next().await { let key = key?; // … } ``` | Method | Iterates over | | --- | --- | | `generic().scan::(options)` | The database's keys | | `hash().hscan::(key, options)` | A hash's fields, each a `(field, value)` pair | | `set().sscan::(key, options)` | A set's members | | `sorted_set().zscan::(key, options)` | A sorted set's members, each a `(member, score)` pair | `ScanOptions::pattern` keeps only what matches a glob-style pattern, and `ScanOptions::count` hints how many elements to look at per round trip. As with Redis's `SCAN`, an element present for the whole iteration is returned at least once, possibly more. ## Publishing to channels The `pubsub()` group is Redis's own channel messaging: a message reaches whoever is subscribed at that moment, and nobody else. For messages that are kept until they're handled, use [Pub/Sub topics](https://encore.dev/docs/rs/primitives/pubsub) instead. ```rust // Returns how many subscribers received the message. SESSIONS.pubsub().publish("room:1", "hello").await?; let mut room = SESSIONS.read().pubsub().subscribe("room:1").await?; while let Some(message) = room.next().await { let text: String = message.payload()?; println!("{}: {text}", message.channel); } ``` `subscribe` takes one channel or several, and `psubscribe` glob-style patterns. Each subscription has a connection of its own, which is closed when the subscription is dropped. A message's `payload` field holds the bytes as sent, and its `payload::()` method converts them. ## Keys Every key is stored under a prefix of the database's own. That's what keeps databases sharing a server apart, and gives each test a database of its own. The prefix is added to every key a command takes and removed from every key a command returns, so your code never sees it. Two things follow: * A script may only touch the keys it's passed through `KEYS`: a key built inside the script doesn't get the prefix. * A client other than this SDK doesn't see the keys it writes under the same names. See below. ## Using another Redis client `connection_string()` returns a URL to connect to the database with another client: ```rust let url = SESSIONS.connection_string()?; ``` That client connects past the database's key prefix: what it stores under a name is not what this SDK reads under that name. Calling `connection_string` grants the service read and write access to the database. ## Using a database from another service In a service other than the one that declares a database, bind it by name with `Database::named`: ```rust use encore::storage::redis::Database; static SESSIONS: Database = Database::named("sessions"); ``` The name must be a string literal. A named database takes no options: it's configured where it's declared. ## Errors Commands return `Result<_, encore::Error>`. A command that fails, or a server that can't be reached, is `ErrCode::Unavailable`. A reply that doesn't convert to the type you asked for is `ErrCode::Internal`. ## What's not supported * Cache clusters and keyspaces, as in Encore.go and Encore.ts. Declare a database and choose your keys. * Commands outside the groups above, including blocking commands such as `BLPOP` and server-wide commands such as `FLUSHALL` or `KEYS`. Use a script, a scan, or another client through `connection_string`. * Transactions, pipelines and `watch` on a `ReadOnlyDatabaseRef`. ## Testing Under `encore test`, each test gets a Redis database of its own, empty unless a fixture fills it, so tests can run in parallel without seeing each other's keys. See [Testing](https://encore.dev/docs/rs/develop/testing). --- Source: https://encore.dev/docs/rs/primitives/secrets Section: Building (Encore.rs) # Storing Secrets and API keys > Keep credentials out of your code API keys, database passwords, and private keys don't belong in source code. Encore.rs lets you declare a secret in code, store its value with Encore's secrets manager, and read it at runtime. ## Declaring a secret Declare a secret as a module-level `static`, then read it with `value()`: ```rust -- billing/src/stripe.rs -- use encore::secrets::Secret; static STRIPE_KEY: Secret = Secret::new("StripeKey"); fn authorization() -> Result { Ok(format!("Bearer {}", *STRIPE_KEY.value()?)) } ``` Encore reads secret declarations from your source code, so a few rules apply: - A secret must be declared as a module-level `static`, not inside a function. - The name must be a string literal. It's an identifier: a letter or underscore, followed by letters, digits, and underscores. - `new` is `const`, so declaring a secret does nothing at runtime. The value is looked up when you call `value()`. ## Scopes The type you declare a secret with decides who it belongs to: | Type | Scope | |---|---| | `Secret` | Belongs to the service that declares it. Two services can each declare a `Secret` with the same name and have different values. It must be declared inside a service. | | `SharedSecret` | One value for the whole app. Any service can declare it, and each service that reads it is granted access to it. | | `GlobalSecret` | One value for the whole app, available to every service without being granted. | ```rust use encore::secrets::{GlobalSecret, Secret, SharedSecret}; // This service's own signing key. static SIGNING_KEY: Secret = Secret::new("SigningKey"); // Shared with other services that read it. static GREETING: SharedSecret = SharedSecret::new("Greeting"); // Available everywhere. static LICENSE: GlobalSecret = GlobalSecret::new("License"); ``` Because a `Secret` is service-scoped, adding one to a service can never collide with, or silently reuse, a name another service already uses. ## Reading values `value()` returns `Result, encore::Error>`. A `Value` is a cheap, cloneable handle that dereferences to the secret's value: ```rust let key = STRIPE_KEY.value()?; let header = format!("Bearer {}", *key); assert!(key.starts_with("sk_")); // String methods work directly ``` A secret's value can change while the app runs. A handle keeps the value it was made with, and the next call to `value()` returns the new one. Call `value()` where you use the secret rather than storing a handle for the lifetime of the process. `Value`'s `Debug` implementation prints `Value()`, so a handle that ends up in a log line or a panic message doesn't leak the secret. ### Errors `value()` returns an error when: - The secret has no value in the current environment (`ErrCode::NotFound`). - The stored bytes can't be read as the secret's type, such as invalid UTF-8 for a `String` or malformed JSON for a `Json` (`ErrCode::Internal`). The error message says where the JSON is wrong, never what it contains. ## Value types A secret is a `String` unless you give it another type: ```rust use encore::secrets::{GlobalSecret, Json, Secret}; use serde::Deserialize; // Raw bytes. static SIGNING_KEY: Secret> = Secret::new("SigningKey"); // A JSON document, deserialized with serde. #[derive(Deserialize)] struct Limits { max_users: u32, } static LIMITS: GlobalSecret> = GlobalSecret::new("Limits"); fn max_users() -> Result { // Json dereferences to the T inside. Ok(LIMITS.value()?.max_users) } ``` The built-in types are `String`, `Vec`, and `Json` for any `T: serde::de::DeserializeOwned`. To read a secret as a type of your own, implement the `SecretValue` trait's `from_bytes(bytes: &[u8]) -> Result`. ## Setting secret values Use the `encore secret` command to set values. The value is read from stdin, so you can pipe it in, or you're prompted for it without echo when running interactively. ### On your machine By default, `encore secret set` stores the value on your machine only, in `.secrets.local.toml` in the app root: ```shell $ encore secret set StripeKey ``` A bare name refers to the app-wide secret (a `SharedSecret` or `GlobalSecret`). A service's own `Secret` is named with the service, either as `service.Name` or with `--service`: ```shell $ encore secret set billing.SigningKey $ encore secret set --service billing SigningKey ``` The name is checked against the secrets your app declares. Run `encore secret set` without a name to be prompted for each declared secret that has no local value yet. The file is a plain TOML file, and you can also edit it by hand. App-wide secrets go in `[shared]`, and each service's own in `[services.]`: ```toml -- .secrets.local.toml -- [shared] StripeKey = "sk_test_123" [services.billing] SigningKey = "local-signing-key" ``` Local values are never uploaded, and they take precedence over values from Encore Cloud. Keep the file out of version control; `encore secret set` adds it to `.gitignore` for you. ### For your team and deployed environments To store a value in Encore Cloud, name the environment types it applies to with `--type`, or specific environments with `--env`: ```shell $ encore secret set --type production StripeKey $ encore secret set --type development,preview,local StripeKey $ encore secret set --env staging StripeKey ``` Environment types are `production`, `development`, `preview`, and `local` (shorthands: `prod`, `dev`, `pr`). A value set for a named environment takes precedence over the value for its type. Values for `local` are synced to every teammate's `encore run` and `encore test`. > **Note** > > Encore Cloud doesn't store per-service secret values yet. A service's own `Secret` can only be given a value on your machine; use a `SharedSecret` or `GlobalSecret` for values that deployed environments need. ### Listing and removing values `encore secret list` shows which secrets have values and where, without printing the values. `encore secret list --missing` lists the declared secrets that have no value and exits non-zero if there are any, which is useful in CI. `encore secret unset` removes a value. It takes the same flags as `set`: without a flag it removes your local value, and `--type` or `--env` removes a value from Encore Cloud. ## Secrets in config A field of a [config](https://encore.dev/docs/rs/develop/config) type can read a secret instead of holding a value: ```rust #[derive(serde::Deserialize, encore::config::ConfigType)] struct Stripe { endpoint: String, #[config(secret = "StripeKey")] api_key: encore::secrets::Secret, } ``` The secret takes the config's scope and must also be declared in code. See [Configuration](https://encore.dev/docs/rs/develop/config#secrets) for details. ## Testing Under `encore test`, a test can give a secret a value of its own with `encore::testing::set_secret`. No other test sees it, and it's gone when the test ends: ```rust #[encore::test] async fn uses_the_test_key() -> Result<(), encore::Error> { encore::testing::set_secret(&STRIPE_KEY, "sk_test_fake")?; assert_eq!(*STRIPE_KEY.value()?, "sk_test_fake"); // Any secret type works: the value is the raw bytes. encore::testing::set_secret(&SIGNING_KEY, [1, 2, 3])?; encore::testing::set_secret(&LIMITS, r#"{"max_users": 2}"#)?; Ok(()) } ``` The value is given as bytes (anything that implements `AsRef<[u8]>`) and read as the secret's type when you call `value()`, so a value that doesn't parse makes `value()` return an error. See [Testing](https://encore.dev/docs/rs/develop/testing) for more. ## How secrets are stored Values you set in Encore Cloud are stored encrypted. When you deploy to your own cloud account on AWS or GCP, Encore replicates them to a secrets manager in that account and provides them to your services at runtime. For local development, `encore run` fetches the `local` values from Encore Cloud and layers your `.secrets.local.toml` on top. --- Source: https://encore.dev/docs/rs/develop/metadata Section: Building (Encore.rs) # Metadata > Get information about the app and the environment it runs in Encore tries to provide a cloud-agnostic environment, but sometimes it's useful to know more about where your code is running. Encore.rs exposes metadata about the application, its environment, build, and deployment through `encore::env::app_meta()`, also available as `encore::app_meta()`. ## Application metadata `app_meta()` returns a `&'static AppMeta`. It's computed once, when the runtime is initialized, so repeated calls are free. ```rust let md = encore::app_meta(); println!("running {} in {}", md.app_id, md.environment.name); ``` `AppMeta` has these fields: | Field | Type | Description | |---|---|---| | `app_id` | `String` | The application's ID on Encore Cloud. Empty if the app isn't linked to Encore Cloud. | | `app_slug` | `String` | The application's human-readable identifier, as shown in the Encore Cloud dashboard. Empty if the app isn't linked. | | `gateways` | `HashMap` | The API gateways that serve the application, keyed by name. | | `environment` | `EnvironmentMeta` | The environment the app is running in. | | `build` | `BuildMeta` | The build: `revision` is the version control revision it was built from. | | `deploy` | `DeployMeta` | The deployment: its `id`, `deploy_time`, and `hosted_services` (the services this deployment hosts, each with a `name`). | ### Environment `md.environment` is an `EnvironmentMeta` with: - `name`: the environment's name. For `encore run` it's `local`, and for `encore test` it's `test`. - `r#type`: an `EnvironmentType`, one of `Production`, `Development`, `Ephemeral` (such as preview environments), or `Test`. `encore run` reports `Development`. - `cloud`: a `CloudProvider`, one of `AWS`, `GCP`, `Azure`, `Encore`, or `Local`. Locally it's `Local`. These enums don't implement `PartialEq`, so compare them with `matches!` or a `match`: ```rust use encore::env::EnvironmentType; let is_prod = matches!(encore::app_meta().environment.r#type, EnvironmentType::Production); ``` ### Gateways An app can have several gateways, so there's no single base URL. Look up the one you mean by name; each `GatewayMeta` has a `name` and a `base_url`: ```rust if let Some(gw) = encore::app_meta().gateway("api") { println!("api is served at {}", gw.base_url); } ``` A process that no gateway serves, such as a worker, has an empty `gateways` map. ## Example use cases ### Using cloud-specific services Encore doesn't natively support every service each cloud offers. Using the environment's cloud provider, you can choose an implementation per cloud. For example, to write audit logs to a data warehouse: ```rust use encore::env::CloudProvider; async fn audit(action: &str, user_id: &str) -> Result<(), AuditError> { match encore::app_meta().environment.cloud { CloudProvider::AWS => write_to_redshift(action, user_id).await, CloudProvider::GCP => write_to_bigquery(action, user_id).await, CloudProvider::Local => write_to_file(action, user_id).await, CloudProvider::Azure | CloudProvider::Encore => Err(AuditError::Unsupported), } } ``` ### Checking the environment type When building a signup flow, you may want to skip email verification during development and in tests: ```rust use encore::env::EnvironmentType; async fn finish_signup(user_id: &str) -> Result<(), SignupError> { match encore::app_meta().environment.r#type { EnvironmentType::Development | EnvironmentType::Test => { mark_email_verified(user_id).await } _ => send_verification_email(user_id).await, } } ``` For values that differ per environment, consider [configuration](https://encore.dev/docs/rs/develop/config) instead: overlay files per environment type or environment name keep the differences out of your code. --- Source: https://encore.dev/docs/rs/develop/testing Section: Building (Encore.rs) # Testing > Every test with infrastructure of its own Encore.rs tests are ordinary Rust tests: `#[test]`, `#[tokio::test]`, or `#[encore::test]`. Run them with `encore test`, and each test gets infrastructure of its own: a migrated database no other test writes to, an empty bucket, an empty Redis database, and topics nobody else publishes to. Tests can run in parallel without seeing each other's data, and need no cleanup. ```rust #[cfg(test)] mod tests { use super::*; #[tokio::test] async fn adds_a_user() -> Result<(), encore::Error> { add_user("ada").await?; let row = DB.query_one("SELECT count(*) FROM users", &[]).await?; assert_eq!(row.get::<_, i64>(0), 1); Ok(()) } } ``` ## Running tests `encore test` runs every service's tests, several services at a time. For a Rust service it runs `cargo test` in the service's directory, with the SDK's `local` feature: ```shell encore test # every service encore test --services users # some services: an expression like `users | billing` encore test --concurrency 2 # at most two services at a time ``` Pass a command to run it instead of the services' own. Without `--services` it runs once, in the current directory; with `--services` it runs in each selected service's directory: ```shell encore test cargo test --features encore-dev/local adds_a_user encore test --services users cargo test --features encore-dev/local -- --nocapture ``` A command you pass gets no features from Encore, so name `encore-dev/local` yourself, or buckets and Pub/Sub won't work. Run tests with `encore test`, not plain `cargo test`: the SDK needs the infrastructure `encore test` provides, and a test binary started without it can't initialize the SDK and exits. ## What each test gets Encore provisions a test's resources the first time the test uses them, and releases them when the test ends. | Resource | In each test | |---|---| | [Database](https://encore.dev/docs/rs/primitives/databases) | A database of its own, cloned from one with every migration applied. It holds what the migrations insert, and nothing else. | | [Bucket](https://encore.dev/docs/rs/primitives/object-storage) | An empty bucket of its own. | | [Redis database](https://encore.dev/docs/rs/primitives/redis) | An empty database of its own, with its own keys and channels. | | [Pub/Sub topic](https://encore.dev/docs/rs/primitives/pubsub) | A topic of its own. No subscription runs: see [Testing Pub/Sub](#testing-pubsub). | | [Secret](https://encore.dev/docs/rs/primitives/secrets) | The app's local value, unless the test sets one with [`set_secret`](#secrets-and-config). | | [Config](https://encore.dev/docs/rs/develop/config) | The app's value, unless the test sets one with [`set_config`](#secrets-and-config). | | [Cron job](https://encore.dev/docs/rs/primitives/cron-jobs) | Never scheduled. Call what serves the job's path. | `connection_string()` on a database returns the connection string of the test's own database, so a client such as sqlx sees the same data the test does. Under `encore test` no other service runs, so `encore::services::base_url` panics. Test a service's HTTP handlers in-process, by calling them or by sending requests to your router directly. ## Which test a call belongs to Encore works out which test a call belongs to from the thread it is made on: - The test's own thread, which the test harness names after the test. - A thread of the Tokio runtime the test runs in, so `tokio::spawn` and `spawn_blocking` from a `#[tokio::test]` are part of the test. A thread the test starts itself is neither, and using a resource there is an error. Make the thread part of the test by entering the test's scope on it: ```rust #[tokio::test] async fn counts_on_another_thread() -> Result<(), encore::Error> { let scope = encore::testing::current()?; let thread = std::thread::spawn(move || { let _guard = scope.enter(); tokio::runtime::Builder::new_current_thread() .enable_all() .build() .unwrap() .block_on(DB.query_one("SELECT count(*) FROM users", &[])) .map(|row| row.get::<_, i64>(0)) }); // Wait for it without blocking the test's own runtime. let count = tokio::task::spawn_blocking(move || thread.join().unwrap()) .await .unwrap()?; assert_eq!(count, 0); Ok(()) } ``` - `encore::testing::current()` returns a `TestScope` for the current test. It is `Clone` and `Send`, and fails outside `encore test`. - `TestScope::enter()` makes the calling thread part of the test until the returned guard is dropped. - `TestScope::scope(future)` runs a future as part of the test on whatever thread polls it. Use it for work spawned onto a runtime that several tests share. - `TestScope::name()` returns the test's name as the harness reports it, such as `tests::adds_a_user`. A test's infrastructure lives as long as the test, or a `TestScope` for it, does. ## `#[encore::test]` `#[encore::test]` is `#[tokio::test]` that tells Encore about the test before it runs, instead of leaving Encore to work it out at the first call. Its runtime's threads are part of the test from the start, its infrastructure is released as soon as it returns, and it loads its module's [fixture](#a-modules-fixture). ```rust #[encore::test] async fn adds_a_user() -> Result<(), encore::Error> { // … Ok(()) } #[encore::test(flavor = "multi_thread", worker_threads = 2)] async fn handles_concurrent_writes() { // … } ``` It takes the same `flavor` (`"current_thread"`, the default, or `"multi_thread"`) and `worker_threads` as `#[tokio::test]`, and a `fixture` argument. It also works on a test that isn't `async`, and keeps the test's other attributes, such as `#[should_panic]`. ## Testing Pub/Sub Under `encore test` nothing handles what a test publishes. Read back what the test, and the code it called, published to a topic with `encore::testing::published`, and call a subscription's handler yourself when you want it to run: ```rust static USER_ADDED: Topic = Topic::new("user-added"); #[encore::pubsub::register] static GREET: Subscription = Subscription::new(&USER_ADDED, "greet", greet); async fn greet(msg: Message) -> Result<(), encore::Error> { // … Ok(()) } #[tokio::test] async fn greets_new_users() -> Result<(), encore::Error> { add_user("ada").await?; let published = encore::testing::published(&USER_ADDED)?; assert_eq!(published.len(), 1); assert_eq!(published[0].data.name, "ada"); greet(published[0].clone()).await?; Ok(()) } ``` `published` returns the messages in the order they were published, each a `Message` with its `data`, `raw_data`, `id`, `attributes` and, on an ordered topic, `ordering_key`. ## Secrets and config A test can give a secret or a config a value of its own. No other test sees it, and it is gone when the test ends: ```rust #[tokio::test] async fn uses_the_test_key() -> Result<(), encore::Error> { encore::testing::set_secret(&API_KEY, "test-key")?; assert_eq!(*API_KEY.value()?, "test-key"); encore::testing::set_config(&CHECKOUT, serde_json::json!({ "max_items": 2 }))?; assert_eq!(CHECKOUT.get().max_items, 2); Ok(()) } ``` `set_secret` takes the value as bytes or a string. For a `Secret>`, pass the JSON. `set_config` checks the value against the config's schema and reads it as the config's type, and returns an error if it doesn't fit. ## Fixtures A test starts from freshly provisioned resources: migrated but otherwise empty databases, empty buckets and empty Redis databases. A fixture is a state to start from instead. Its setup is ordinary app code that runs once per test run, and each test that loads the fixture gets its own copy of what the setup left, to change as it likes. Declare fixtures with `encore::fixture!`, and load one at the start of a test: ```rust #[cfg(test)] mod tests { use super::*; encore::fixture! { /// Two users and their roster. static TEAM = async { add_user("ada").await?; add_user("grace").await?; UPLOADS.upload("roster.txt", b"ada,grace".to_vec()).await?; Ok(()) }; } #[tokio::test] async fn lists_the_team() -> Result<(), encore::Error> { TEAM.load().await?; assert_eq!(list_users().await?.len(), 2); Ok(()) } } ``` The setup is an expression producing a future, such as an `async` block or a call to an `async fn`, that returns `Result<(), encore::Error>`. Rules to know: - **Load first.** Call `load()` before the test first uses a resource. Once a test has a database, bucket or Redis database, what it started from can't change, and `load()` returns an error. - **What is saved.** Databases, buckets and Redis databases are saved. A resource the setup didn't touch starts as it would without the fixture. Topics, secrets and config start each test as usual. - **When the setup runs.** The first test of a run to load a fixture builds it, and the others wait for it, across threads and test binaries. Encore keeps a fixture between runs while the app's source is unchanged, unless its setup touched a Redis database: such a fixture is built again each run. - **Names.** `fixture!` names each fixture after its module and itself. Two fixtures of the same name are one fixture. ### Extending a fixture A fixture declared `static NAME: PARENT` starts its setup from what `PARENT` saved: ```rust encore::fixture! { static TEAM = async { add_user("ada").await?; add_user("grace").await?; Ok(()) }; /// `TEAM`, and one more. static BIGGER_TEAM: TEAM = async { add_user("linus").await?; Ok(()) }; } ``` ### A module's fixture `for mod` makes a fixture the one every `#[encore::test]` in its module, and in the modules inside it, loads before it runs. The innermost module's fixture wins, and a test can choose another, or none: ```rust mod with_team { use super::*; encore::fixture! { static SEEDED for mod: TEAM = async { add_user("barbara").await?; Ok(()) }; } #[encore::test] async fn starts_from_seeded() { // Three users from TEAM and SEEDED. } #[encore::test(fixture = TEAM)] async fn starts_from_team() { // TEAM's two users. } #[encore::test(fixture = none)] async fn starts_empty() { // No fixture. } } ``` A plain `#[tokio::test]` in such a module loads nothing by itself. Rather than run it on the wrong data, Encore returns an error at its first use of a resource until it calls `SEEDED.load().await?` or `encore::testing::without_fixture()?`. ### Declaring a fixture by hand `fixture!` declares a `Fixture`. You can declare one directly, which suits a fixture shared between modules: ```rust use encore::testing::Fixture; pub static SEEDED: Fixture = Fixture::new("seeded", || Box::pin(seed())); pub static INVOICED: Fixture = Fixture::extending(&SEEDED, "invoiced", || Box::pin(invoice())); async fn seed() -> Result<(), encore::Error> { DB.execute("INSERT INTO users (name) VALUES ('ada'), ('grace')", &[]).await?; Ok(()) } async fn invoice() -> Result<(), encore::Error> { // … Ok(()) } ``` The name identifies the fixture across the app's test binaries. --- Source: https://encore.dev/docs/rs/observability/logging Section: Building (Encore.rs) # Logging > Structured logging helps you understand your application Encore.rs logs through the standard [`log`](https://docs.rs/log) crate. Install Encore's logger once at startup, and every `log` record is printed as a structured line, with its key-value pairs as fields and the IDs of the current trace and span, so it's linked to the [trace](https://encore.dev/docs/rs/observability/tracing) it was written in. ## Setup Add the `log` crate with its `kv` feature, for key-value pairs: ```toml -- Cargo.toml -- [dependencies] encore-dev = { path = "/path/to/encore-dev" } log = { version = "0.4", features = ["kv"] } ``` Call `encore::log::init()` at the start of `main`: ```rust -- src/main.rs -- #[tokio::main] async fn main() { encore::initialize(); encore::log::init(); // ... } ``` `init()` installs Encore's logger as the `log` crate's global logger. It's safe to call more than once, but it panics if another global logger is already installed, such as `env_logger`. ## Usage Use the `log` macros: `error!`, `warn!`, `info!`, `debug!`, and `trace!`. Key-value pairs go before the message, separated from it by a semicolon: ```rust log::info!( order_id = order.id.as_str(), customer_id = customer.id.as_str(), total = order.total; "order created" ); ``` Each pair becomes a field of the log record. Pass variable data as fields rather than interpolating it into the message: ```rust // Prefer log::info!(order_id = order_id.as_str(); "order created"); // Avoid log::info!("order {order_id} created"); ``` Interpolated values have to be parsed back out before they can be queried, and they make every message unique, so equivalent events can no longer be grouped or counted. ### Logging errors Log an error as a field. The `kv` feature captures values that implement `Display` with the `%` sigil: ```rust if let Err(err) = charge_customer(&customer_id, amount).await { log::error!( err:% = err, customer_id = customer_id.as_str(), amount = amount; "payment authorization failed" ); return Err(err); } ``` ## What's in a record Each record is printed to stderr in Encore's structured log format, where `encore run` and Encore's log collection pick it up. A record has: - The level, message, and time. - The key-value pairs as fields. - The record's target as its logger name: by default the module path, such as `my_app::orders`. - The source file and line, as `code.file.path` and `code.line.number`. - The trace ID and span ID of the current OpenTelemetry context, if there is one. The span IDs come from `encore::opentelemetry::Context::current()`. Inside a Pub/Sub subscription handler, Encore sets that context for you. In an HTTP handler, attach your request span's context to the handler's future, as [Tracing](https://encore.dev/docs/rs/observability/tracing#parenting-on-your-request-span) shows, and records written while handling the request are linked to its trace. ## Levels By default, records at `info`, `warn`, and `error` are printed, and `debug` and `trace` records are dropped. The SDK's own log lines (from the `encore` crate and the runtime) are logged under the `encore.runtime` logger. Their level is controlled with the `RUST_LOG` environment variable, using the same syntax as `env_logger`. ## Using your own logger If your app installs its own global logger, compose Encore's into it instead of calling `init()`. `encore::log::logger()` returns Encore's logger as a `&'static impl log::Log`, which you can forward records to. ## `log` and `tracing` Encore's logger reads records from the `log` crate. Events from the [`tracing`](https://docs.rs/tracing) crate's macros aren't log records, so they aren't printed by it. Use `tracing` for spans (see [Tracing](https://encore.dev/docs/rs/observability/tracing)) and `log` for log lines, or enable the `tracing` crate's `log-always` feature, which also emits its events as `log` records when a `tracing` subscriber is installed. ## What to log - A business state transition, such as a subscription activating or an order shipping. - A decision the application made, together with the inputs that drove it. - A fallback or retry that allowed an operation to complete. - A failure, at the boundary that owns the recovery decision. - A security-relevant or administrative action. Encore already traces database queries, Redis commands, object storage operations, and Pub/Sub messages. Logs that narrate the same steps ("querying database", "publishing event") duplicate the trace. Use a [metric](https://encore.dev/docs/rs/observability/metrics) rather than a log line per event when you only need to know how often something happens. ## Viewing logs Logs are shown in `encore run`'s output. To read them afterwards, use `encore logs`, which reads the app's local telemetry store, so the app doesn't need to be running: ```shell $ encore logs # the latest session $ encore logs --errors # only entries in which something failed $ encore logs --svc orders --since 10m ``` --- Source: https://encore.dev/docs/rs/observability/metrics Section: Building (Encore.rs) # Metrics > Track custom metrics in your Rust application Encore.rs lets you declare custom metrics in your code. They're exported along with Encore's built-in metrics, displayed in the Encore Cloud dashboard, and can be sent on to third-party observability services. See the [Platform metrics documentation](https://encore.dev/docs/platform/observability/metrics) for integrations with services like Grafana Cloud and Datadog. ## Defining metrics Declare a metric as a module-level `static`, and record to it wherever the thing it measures happens. Its doc comment is its description: ```rust -- orders/src/metrics.rs -- use encore::metrics::{Counter, Gauge, Histogram, UpDownCounter}; /// Orders placed. static ORDERS: Counter = Counter::new("orders_placed").unit("{order}"); /// Jobs waiting for a worker. static QUEUED: UpDownCounter = UpDownCounter::new("jobs_queued").unit("{job}"); /// The temperature of the oven. static OVEN: Gauge = Gauge::new("oven_temperature").unit("Cel"); /// How long an order takes to place. static LATENCY: Histogram = Histogram::new("order_latency").unit("s"); async fn place_order(order: &Order) -> Result<(), encore::Error> { let started = std::time::Instant::now(); // ... ORDERS.increment(); LATENCY.record(started.elapsed().as_secs_f64()); Ok(()) } ``` Encore reads metric declarations from your source code, so a few rules apply: - A metric must be declared as a module-level `static`. - The name must be a string literal: a letter, then letters, digits, `_`, `.`, `-`, and `/`. - The options (`.unit(…)`, `.description(…)`, and for histograms `.buckets(…)`) are chained directly onto `new`, each at most once, with literal arguments. The unit is exported as given. [UCUM](https://ucum.org/ucum) is the convention: `"s"`, `"By"`, `"{request}"`. If a metric has no doc comment, give it a description with `.description("…")`. ## Metric types | Type | What it is | Recording | Value types | |---|---|---|---| | `Counter` | A total that only goes up: how many times something happened, or how much of something there has been. | `increment()`, `add(n)` | `u64` (default), `f64` | | `UpDownCounter` | A total that goes up and down: requests in flight, jobs queued. | `increment()`, `decrement()`, `add(n)` | `i64` (default), `f64` | | `Gauge` | What something is right now. | `set(v)` | `f64` (default), `i64`, `u64` | | `Histogram` | How recorded values are distributed: the median latency, the 99th percentile. | `record(v)` | `f64` | Pick a value type other than the default with the type parameter: ```rust /// Bytes received. static RECEIVED: Counter = Counter::new("bytes_received").unit("By"); /// Connections open right now. static CONNECTIONS: Gauge = Gauge::new("connections_open"); ``` A counter doesn't go down: `add` ignores an amount below zero. ### Histogram buckets A histogram counts what it records in buckets. Set their upper bounds, in increasing order, with `.buckets(…)`. A value is counted in the first bucket whose bound it's no greater than, or in one past the last. Without `.buckets`, the OpenTelemetry SDK's default buckets are used. ```rust /// How long adding a user takes. static ADD_DURATION: Histogram = Histogram::new("add_user_duration") .unit("s") .buckets(&[0.001, 0.01, 0.1, 1.0]); ``` ## Labels A metric group records one number for each combination of its labels. Define the labels as a struct that derives `Labels`; each field is a label, named after the field: ```rust use encore::metrics::{CounterGroup, Labels}; #[derive(Labels)] struct Outcome { method: &'static str, status: u16, } /// Requests handled, by method and status. static REQUESTS: CounterGroup = CounterGroup::new("requests_handled"); fn record(method: &'static str, status: u16) { REQUESTS.with(&Outcome { method, status }).increment(); } ``` Every metric type has a group: `CounterGroup`, `UpDownCounterGroup`, `GaugeGroup`, and `HistogramGroup`. `with(&labels)` returns the metric for those labels. Making it converts the labels to attributes, so keep the result around where you record to it often. The label type must be named in the static's type, as in `CounterGroup`, and a group takes a value type as its second parameter: `GaugeGroup`. Label values can be strings (`&'static str`, `String`, `Cow`), integers, `bool`, and `char`. To use your own type, such as an enum, implement `LabelValue`: ```rust use encore::metrics::LabelValue; use encore::opentelemetry::Value; enum Plan { Free, Paid, } impl LabelValue for Plan { fn label_value(&self) -> Value { match self { Plan::Free => "free", Plan::Paid => "paid", } .into() } } ``` > **Important** > > Each distinct combination of label values is a separate time series. Use labels for a small set of values, like a status or a region, never for something like a user ID. For attributes of any other shape, record with the [OpenTelemetry API](#using-the-opentelemetry-api). ## Scopes As with [secrets](https://encore.dev/docs/rs/primitives/secrets#scopes), the type you declare a metric with decides who records to it: | Types | Scope | Exported as | |---|---|---| | `Counter`, `Gauge`, … | Belongs to the service that declares it, which alone records to it. Two services can each declare a metric with the same name. It must be declared inside a service. | `.`, e.g. `shop.orders_placed` | | `SharedCounter`, `SharedGauge`, … | Shared between services under one name. Each service that records to it is recorded as using it. | `` | | `GlobalCounter`, `GlobalGauge`, … | The whole app's. Any code can record to it, including a library outside every service, and who does isn't recorded. | `` | Groups come in the same three scopes: `SharedCounterGroup`, `GlobalHistogramGroup`, and so on. ```rust use encore::metrics::{GlobalHistogram, SharedCounterGroup}; /// Attempts to add a user, by how they went. static ADD_ATTEMPTS: SharedCounterGroup = SharedCounterGroup::new("add_user_attempts"); /// How long adding a user takes. static ADD_DURATION: GlobalHistogram = GlobalHistogram::new("add_user_duration").unit("s"); ``` ### Recording to a metric declared elsewhere A shared or global metric is declared once in the app. Other code in the same crate records to it by importing the declaration. From another service's crate, or a service written in another language, bind it by name with `named`: ```rust use encore::metrics::CounterGroup; // Declared as a SharedCounterGroup in another service. static ADD_ATTEMPTS: CounterGroup = CounterGroup::named("add_user_attempts"); ``` `named` is available on the service-scoped types (`Counter::named`, `CounterGroup::named`, and so on). The binding must be of the same kind and labels as the declaration, and it takes the declaration's scope, unit, description, and buckets, so it has no options of its own. It's exported under the name as written, never prefixed with a service name. ### Passing a metric to other code Encore sees which service records to a metric from where it's used by name. To hand a metric to a function that records to whichever metric it's given, pass `.reference()`, which counts as recording to it: ```rust fn time(histogram: &Histogram, f: F) { let started = std::time::Instant::now(); f(); histogram.record(started.elapsed().as_secs_f64()); } time(LATENCY.reference(), || place_order_sync()); ``` ## Using the OpenTelemetry API Each Encore metric is an instrument of the [OpenTelemetry Metrics API](https://docs.rs/opentelemetry/0.29/opentelemetry/metrics/), from the meter `encore`. You can also create instruments with the OpenTelemetry API directly, for attributes of any value or for observable instruments. Encore's meter provider, `encore::metrics::meter_provider()`, exports them alongside Encore's metrics. The OpenTelemetry crate is re-exported as `encore::opentelemetry`. Encore doesn't install a global meter provider. Install its provider at startup to use `global::meter`: ```rust use std::sync::LazyLock; use encore::opentelemetry::{global, metrics, KeyValue}; /// Page views, by page. static PAGE_VIEWS: LazyLock> = LazyLock::new(|| { global::meter("my-app") .u64_counter("page_views") .with_unit("{view}") .build() }); #[tokio::main] async fn main() { encore::initialize(); global::set_meter_provider(encore::metrics::meter_provider()); // ... } fn viewed(page: &str) { PAGE_VIEWS.add(1, &[KeyValue::new("page", page.to_string())]); } ``` Or get a meter from the provider directly, and pass the provider to instrumentation libraries that take one: ```rust use encore::opentelemetry::metrics::MeterProvider as _; let meter = encore::metrics::meter_provider().meter("my-app"); ``` Instruments made this way aren't declared to Encore: they have no scope and aren't prefixed with a service name. ## Viewing metrics Locally, `encore metrics list` shows the metrics reported in the last hour, with their stats over a window, and `encore metrics get ` shows one metric's series. Add `--watch` to keep the output updated as new data is exported: ```shell $ encore metrics list $ encore metrics get shop.orders_placed --since 15m $ encore metrics get requests_handled --attr status=500 --watch ``` In deployed environments, metrics are shown in the Encore Cloud dashboard. See the [Platform metrics documentation](https://encore.dev/docs/platform/observability/metrics). ## Testing Under `encore test`, as anywhere the runtime exports no metrics, recording to a metric does nothing. Calls like `increment()` and `record()` are safe to leave in code under test. --- Source: https://encore.dev/docs/rs/observability/tracing Section: Building (Encore.rs) # Tracing > Get your own spans into Encore's traces Encore.rs records [distributed traces](https://encore.dev/docs/tracing) of what your app does with its infrastructure. Since Encore.rs doesn't own your HTTP server, your request handling is traced with your own instrumentation, which Encore.rs supports through [OpenTelemetry](https://opentelemetry.io). This page shows how to connect the two, so a request's trace shows both the request and the queries, messages, and other calls it made. ## What Encore traces automatically Encore records a span for each: - Postgres query and transaction - Redis command - Object storage operation - Pub/Sub publish, and each message delivered to a subscription Requests routed through Encore's API gateway also get a span from the gateway, and reach your service with a W3C `traceparent` header so your own request span can continue the trace. Each infrastructure span is parented on the current OpenTelemetry context, `encore::opentelemetry::Context::current()`. Inside a Pub/Sub subscription handler, Encore sets that context to the message delivery's span for you. Everywhere else, including your HTTP handlers, the context is whatever your code has attached. If nothing is attached, each query or publish starts a trace of its own. Log records written with [`encore::log`](https://encore.dev/docs/rs/observability/logging) carry the IDs of the current context's span too. ## The OpenTelemetry API Encore.rs records spans through its own OpenTelemetry tracer provider, and re-exports the `opentelemetry` crate (version 0.29) as `encore::opentelemetry`, so you use the same version as Encore. The `encore::telemetry` module gives you: | Function | Returns | |---|---| | `tracer_provider()` | Encore's `TracerProvider`, which records into Encore's traces. Cheap to clone. | | `tracer(name)` | A `Tracer` from that provider, with the instrumentation scope `name`. | | `parse_traceparent(traceparent, tracestate)` | The remote `SpanContext` described by W3C `traceparent` and optional `tracestate` header values, or `None` if they're malformed. | Encore doesn't install its provider as the OpenTelemetry global. To use libraries that record through the global tracer provider, install it at startup: ```rust encore::opentelemetry::global::set_tracer_provider(encore::telemetry::tracer_provider()); ``` ### Recording a span directly With the OpenTelemetry API, start a span, put it in a context, and run the work under that context: ```rust use encore::opentelemetry::trace::{FutureExt, Tracer as _}; use encore::opentelemetry::Context; async fn charge(order: &Order) -> Result<(), encore::Error> { let tracer = encore::telemetry::tracer("billing"); let cx = Context::current_with_span(tracer.start("charge order")); // The query's span is a child of "charge order", which ends when `cx` is dropped. record_charge(order).with_context(cx).await } ``` ## Tracing an axum app Most Rust web frameworks are instrumented with the [`tracing`](https://docs.rs/tracing) crate. [`tracing-opentelemetry`](https://docs.rs/tracing-opentelemetry) turns `tracing` spans into OpenTelemetry spans, and with Encore's tracer they're recorded into Encore's traces. The setup below uses axum and tower-http's `TraceLayer`, and works the same way for other frameworks built on `tracing`. ### Dependencies `tracing-opentelemetry` 0.30 is the release built on `opentelemetry` 0.29, the version Encore.rs uses: ```toml -- Cargo.toml -- [dependencies] encore-dev = { path = "/path/to/encore-dev" } axum = "0.7" tokio = { version = "1", features = ["rt-multi-thread", "macros", "net"] } tower-http = { version = "0.6", features = ["trace"] } tracing = "0.1" tracing-opentelemetry = { version = "0.30", default-features = false } tracing-subscriber = { version = "0.3", default-features = false, features = ["registry", "std"] } rand = "0.8" ``` ### A tracer for tracing-opentelemetry `tracing-opentelemetry` needs a tracer that also implements its `PreSampledTracer` trait, which reports a span's context before the span is built. Wrap Encore's tracer in one. `Tracer::span_context` does the work: it fills in the span's IDs and applies Encore's sampling decision. ```rust -- src/otel.rs -- use std::borrow::Cow; use encore::opentelemetry::trace::{SpanBuilder, SpanId, TraceContextExt, TraceId}; use encore::opentelemetry::Context; use tracing_opentelemetry::{OtelData, PreSampledTracer}; /// Encore's tracer, with the `PreSampledTracer` tracing-opentelemetry needs. #[derive(Debug, Clone)] pub struct Tracer(encore::telemetry::Tracer); /// A `Tracer` with instrumentation scope `name`. pub fn tracer(name: impl Into>) -> Tracer { Tracer(encore::telemetry::tracer(name)) } impl encore::opentelemetry::trace::Tracer for Tracer { type Span = ::Span; fn build_with_context(&self, builder: SpanBuilder, parent_cx: &Context) -> Self::Span { self.0.build_with_context(builder, parent_cx) } } impl PreSampledTracer for Tracer { fn sampled_context(&self, data: &mut OtelData) -> Context { let cx = self.0.span_context(&mut data.builder, &data.parent_cx); data.parent_cx.with_remote_span_context(cx) } fn new_trace_id(&self) -> TraceId { loop { let id = TraceId::from_bytes(rand::random()); if id != TraceId::INVALID { return id; } } } fn new_span_id(&self) -> SpanId { loop { let id = SpanId::from_bytes(rand::random()); if id != SpanId::INVALID { return id; } } } } ``` ### Installing the subscriber In `main`, install a `tracing` subscriber with the OpenTelemetry layer and the tracer above. Filter it to your own crate's spans, so spans from libraries such as hyper and h2 stay out of your traces: ```rust -- src/main.rs -- mod otel; use tracing::Level; use tracing_subscriber::filter::Targets; use tracing_subscriber::layer::SubscriberExt; use tracing_subscriber::util::SubscriberInitExt; use tracing_subscriber::Layer; #[tokio::main] async fn main() { encore::initialize(); encore::log::init(); tracing_subscriber::registry() .with( tracing_opentelemetry::layer() .with_tracer(otel::tracer("my_app")) // Leave out the thread and busy/idle time attributes. .with_threads(false) .with_tracked_inactivity(false) .with_filter(Targets::new().with_target("my_app", Level::INFO)), ) .init(); // ... build the router and serve, below. } ``` Replace `my_app` with your crate's name as it appears in module paths (with `_` for `-`). ### A span for each request tower-http's `TraceLayer` creates a span for each request. Make it a server span named after the route, with the HTTP semantic-convention attributes, and continue the caller's trace from its `traceparent` header: ```rust use std::time::Duration; use axum::extract::{MatchedPath, Request}; use axum::http::{header, HeaderMap}; use axum::response::Response; use encore::opentelemetry::trace::TraceContextExt; use encore::opentelemetry::Context; use tracing::{field, Span}; use tracing_opentelemetry::OpenTelemetrySpanExt; /// The server span for a request, continuing the caller's trace. fn request_span(req: &Request) -> Span { let method = req.method().as_str(); let route = req.extensions().get::().map(MatchedPath::as_str); let user_agent = req .headers() .get(header::USER_AGENT) .and_then(|v| v.to_str().ok()); let span = tracing::info_span!( "request", otel.name = format!("{method} {}", route.unwrap_or_default()).trim_end(), otel.kind = "server", otel.status_code = field::Empty, http.request.method = method, http.route = route, url.path = req.uri().path(), user_agent.original = user_agent, http.response.status_code = field::Empty, "error.type" = field::Empty, ); if let Some(parent) = remote_parent(req.headers()) { span.set_parent(parent); } span } /// The caller's span, from the W3C `traceparent` and `tracestate` headers. fn remote_parent(headers: &HeaderMap) -> Option { let header = |name: &str| headers.get(name).and_then(|v| v.to_str().ok()); let cx = encore::telemetry::parse_traceparent(header("traceparent")?, header("tracestate"))?; Some(Context::new().with_remote_span_context(cx)) } /// Records the response's status, and marks server errors. fn record_response(res: &Response, _latency: Duration, span: &Span) { let status = res.status(); span.record("http.response.status_code", status.as_u16()); if status.is_server_error() { span.record("otel.status_code", "ERROR"); span.record("error.type", status.as_str()); } } ``` ### Parenting on your request span `tracing-opentelemetry` doesn't change the current OpenTelemetry context when a `tracing` span is entered. So inside your handler, `Context::current()` isn't the request span, and Encore's query and publish spans wouldn't be its children. Fix that with a middleware that runs each handler under the request span's context, using `FutureExt::with_context`: ```rust use axum::middleware::Next; use encore::opentelemetry::trace::FutureExt; /// Runs the handler under the request span's OpenTelemetry context, so /// infrastructure calls and log lines are parented on it. async fn otel_context(req: Request, next: Next) -> Response { let cx = Span::current().context(); next.run(req).with_context(cx).await } ``` Then put it together. The middleware sits inside `TraceLayer`, so `Span::current()` in it is the request span: ```rust use axum::middleware; use axum::routing::get; use axum::Router; use tower_http::trace::TraceLayer; let app = Router::new() .route("/", get(hello)) .layer(middleware::from_fn(otel_context)) .layer( TraceLayer::new_for_http() .make_span_with(request_span) .on_request(()) .on_response(record_response) .on_failure(()), ); let listener = tokio::net::TcpListener::bind("0.0.0.0:8080").await.unwrap(); axum::serve(listener, app) .with_graceful_shutdown(encore::shutdown::signal()) .await .expect("serve"); std::process::exit(encore::shutdown::drained().await.code()); ``` Awaiting `encore::shutdown::drained()` before exiting also makes sure the last spans are exported. See [Graceful shutdown](https://encore.dev/docs/rs/develop/shutdown). ### Your own spans A span you create in a handler needs the same treatment: attach its context to the future that makes the infrastructure calls, as well as instrumenting the future with the `tracing` span: ```rust use encore::opentelemetry::trace::FutureExt; use tracing::Instrument; use tracing_opentelemetry::OpenTelemetrySpanExt; async fn hello() -> String { let span = tracing::info_span!("load greeting"); let cx = span.context(); let name = latest_user() .with_context(cx) .instrument(span) .await .unwrap_or_else(|| "World".to_string()); format!("Hello, {name}!\n") } ``` The query in `latest_user` is then recorded as a child of `load greeting`, which is a child of the request span. > **Warning** > > Don't attach a context with `Context::attach` and hold the guard across an `.await`. The guard sets a thread-local context, so while your task is suspended, other tasks that run on the same thread would inherit it, and their spans would end up in the wrong trace. Use `FutureExt::with_context`, which sets the context only while that future is being polled. ## Viewing traces Locally, list the most recent traces and print one with the `encore trace` command: ```shell $ encore trace list $ encore trace get ``` `encore trace get` takes the 32-character trace ID shown by `encore trace list`. For deployed environments, traces are shown in the Encore Cloud dashboard. See [Distributed Tracing](https://encore.dev/docs/tracing) for more about tracing in Encore. --- Source: https://encore.dev/docs/rs/develop/config Section: Building (Encore.rs) # Configuration > Define behavior in specific environments Configuration lets you set values your code reads at runtime, such as limits, feature flags, and endpoints, and override them for specific environments without changing the code. In Encore.rs a config is a Rust type. Encore derives a schema from the type, you write the values in TOML or JSON files next to your code, and Encore checks every value against the schema before your service sees it. > **Important** > > Don't put credentials in config files. Store them as [secrets](https://encore.dev/docs/rs/primitives/secrets), and reference them from a config [secret field](#secrets) if needed. ## Declaring a config Define a type that derives serde's `Deserialize` and Encore's `ConfigType`, and declare the config as a module-level `static`: ```rust -- orders/src/checkout.rs -- use encore::config::{Config, ConfigType, Duration}; use serde::Deserialize; #[derive(Deserialize, ConfigType)] struct Checkout { /// Most items one order may hold. #[config(min = 1, max = 500)] max_items: u32, #[serde(default)] promo: Option, payment_timeout: Duration, } #[derive(Deserialize, ConfigType)] struct Promo { code: String, discount: u8, } static CHECKOUT: Config = Config::new("checkout"); fn too_many(items: usize) -> bool { items > CHECKOUT.get().max_items as usize } ``` Then write its value in a file named after the config, next to the service's `encore.service.json`: ```toml -- orders/checkout.config.toml -- max_items = 50 payment_timeout = "1m30s" [promo] code = "FALL26" discount = 10 ``` Encore reads config declarations and their types from your source code, so a few rules apply: - A config must be declared as a module-level `static`, with its value type named in the static's type: `static CHECKOUT: Config`. - The name must be a string literal. It's an identifier: a letter or underscore, followed by letters, digits, and underscores. - The type, and every type it contains, must derive `Deserialize`. Derive `ConfigType` on any struct whose fields use `#[config(...)]`. ## The type is the schema Encore builds a JSON Schema from your type: its fields and their types, serde attributes such as `rename`, `rename_all`, `default`, and `flatten`, enum tagging, the `#[config(...)]` constraints, and doc comments, which become field descriptions. Every value is checked against that schema: while you edit a file under `encore run`, on deploy, and when the service starts. A service won't start with a value it can't read, which is why `get()` can't fail. ### Supported types - `bool`, the integer types up to 64 bits, `f32`, `f64`, `char`, and `String` - `Option`, `Vec`, `VecDeque`, `HashSet`, `BTreeSet` - `HashMap` and `BTreeMap` with string keys - `Box`, `Arc`, `Rc` (including `Box` and `Arc`), and `PathBuf` - `serde_json::Value` and `serde_json::Map` for free-form data - `encore::config::Duration` for lengths of time - Your own structs (with named fields, or newtypes) and enums, except `#[serde(untagged)]` enums Encore reports an error at the part of the type it can't describe, for example a tuple, a borrowed `&str`, or `std::time::Duration`. ### Constraints `#[config(...)]` on a field adds a constraint to the schema: | Option | Applies to | Meaning | |---|---|---| | `min = N`, `max = N` | numbers | The least and greatest value allowed | | `min_len = N`, `max_len = N` | strings | The fewest and most characters | | `pattern = "regex"` | strings | A regular expression the value must match | | `min_items = N`, `max_items = N` | lists | The fewest and most items | | `secret = "Name"` | `Secret` fields | The field reads this secret; see [Secrets](#secrets) | ### Durations `encore::config::Duration` is written as a number and a unit, or several: `"30s"`, `"250ms"`, `"1h30m"`, `"1.5h"`. The units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, and `h`. It dereferences to a `std::time::Duration`, and converts into one with `.into()`: ```rust let timeout: std::time::Duration = CHECKOUT.get().payment_timeout.into(); ``` ### Defaults A field that no file sets takes its default from your code: `#[serde(default)]`, `#[serde(default = "path")]`, or `None` for an `Option`. If no file gives a config a value at all, `get()` returns the defaults. ## Reading a config `get()` returns a `Value`: a snapshot of the config's current value that dereferences to `T`. It's cheap to make and to clone. ```rust let checkout = CHECKOUT.get(); if let Some(promo) = &checkout.promo { apply_discount(&promo.code, promo.discount); } ``` A config's value can change while the app runs. Edit its file under `encore run` and the running service has the new value within a second, without a restart. A snapshot keeps the value it was made with, so hold one for as long as you need a consistent view (for example, one request), and call `get()` again for the next. ### Reacting to changes If you build something from a config, such as an HTTP client or a connection pool, rebuild it when the value changes with `on_change`: ```rust CHECKOUT.on_change(|checkout| { println!("max items is now {}", checkout.max_items); }); ``` The callback is called with each new value from then on. It runs on the runtime's threads, so keep it quick and spawn anything that waits. ## Scopes As with [secrets](https://encore.dev/docs/rs/primitives/secrets#scopes), the type you declare a config with decides who it belongs to: | Type | Scope | Value files live | |---|---|---| | `Config` | Belongs to the service that declares it. Two services can each declare a config with the same name. It must be declared inside a service. | Next to the service's `encore.service.json` | | `SharedConfig` | One value for the whole app. Any service can read it, and each one that does is recorded as using it. | Next to `encore.app` | | `GlobalConfig` | One value for the whole app, available to every service without being recorded. | Next to `encore.app` | ```rust use encore::config::{ConfigType, SharedConfig}; use serde::Deserialize; #[derive(Deserialize, ConfigType)] struct Flags { #[serde(default)] new_checkout: bool, } static FLAGS: SharedConfig = SharedConfig::new("flags"); ``` ## Environment-specific values A config's value is its base file plus any overlay files that apply to the current environment. Each file can be `.toml` or `.json`: | File | Applies to | |---|---| | `checkout.config.toml` | Every environment | | `checkout.production.config.toml` | Environments of a type: `local`, `test`, `development`, `preview`, or `production` | | `checkout.@prod-eu.config.toml` | The environment named `prod-eu` | | `checkout.user.config.toml` | Only your own `encore run` | Overlays are applied in that order, lowest first, on top of the defaults in code. They're merged field by field using [JSON Merge Patch](https://www.rfc-editor.org/rfc/rfc7396): tables merge, any other value replaces the one below it, and `null` removes it. An overlay only needs the fields it changes: ```toml -- orders/checkout.production.config.toml -- max_items = 200 ``` Each layer is checked against the schema on its own, so a mistake is reported in the file that makes it, and then the merged value is checked in full. The user override (`.user.config.toml`) is for your own values on your own machine. It applies to `encore run` only, never to tests or deployed environments. Keep it out of version control: `encore app new` and `encore app init` add `*.user.config.*` to the app's `.gitignore`. ## Secrets A config field can read a [secret](https://encore.dev/docs/rs/primitives/secrets) instead of holding a value. Declare the field as `encore::secrets::Secret` and name the secret with `#[config(secret = "...")]`: ```rust use encore::config::{ConfigType, GlobalConfig}; use encore::secrets::{GlobalSecret, Secret}; use serde::Deserialize; #[derive(Deserialize, ConfigType)] struct Payments { stripe: Stripe, } #[derive(Deserialize, ConfigType)] struct Stripe { endpoint: String, #[config(secret = "StripeKey")] api_key: Secret, } static PAYMENTS: GlobalConfig = GlobalConfig::new("payments"); // The secret is declared in code as any secret is, with the config's scope. static STRIPE_KEY: GlobalSecret = GlobalSecret::new("StripeKey"); fn stripe_key() -> Result { Ok(PAYMENTS.get().stripe.api_key.value()?.to_string()) } ``` - The field's type is always `Secret`; the secret has the config's own scope. A `Config` reads its service's own secret, a `SharedConfig` a shared one, and a `GlobalConfig` a global one. - The secret must also be declared in code (`Secret::new`, `SharedSecret::new`, or `GlobalSecret::new`). A secret field that names an undeclared secret is an error. - No config file sets the field. The secret holds the value, per environment, as secrets do. - A secret field can't be inside a list, a map, or an enum. ## Inspecting and editing config The `encore config` command works with your app's configs: | Command | What it does | |---|---| | `encore config list` | Lists the configs the app declares, and whether each value for `encore run` is valid | | `encore config show ` | Shows a config's effective value and which file and line each value comes from. `--env-type` and `--env` pick the environment. | | `encore config schema ` | Prints the JSON Schema the config's values are checked against | | `encore config validate` | Checks every config file for every environment type, and exits non-zero if any value is invalid | | `encore config set ` | Sets one value, such as `max_items` or `promo.code`, keeping the rest of the file as it is | | `encore config unset ` | Removes one value, so a lower layer or the default applies | | `encore config link` | Adds a schema reference to each config file, so editors check and complete what you write | A service's own config is named `.`, for example `encore config show orders.checkout`. `set` and `unset` write to the user override by default. Pass `--base`, `--env-type `, or `--env ` to change another layer: ```shell $ encore config set orders.checkout max_items 100 $ encore config set orders.checkout max_items 200 --env-type production ``` ## Testing Under `encore test`, a config's value is what the files say for the `test` environment type. Use a `checkout.test.config.toml` overlay to change values for all tests. To give one test a value of its own, use `encore::testing::set_config`. No other test sees it, and it's gone when the test ends. The value is checked against the config's schema: ```rust #[encore::test] async fn rejects_large_orders() -> Result<(), encore::Error> { encore::testing::set_config( &CHECKOUT, serde_json::json!({"max_items": 2, "payment_timeout": "30s"}), )?; assert!(too_many(3)); Ok(()) } ``` See [Testing](https://encore.dev/docs/rs/develop/testing) for more. --- Source: https://encore.dev/docs/rs/develop/shutdown Section: Building (Encore.rs) # Graceful shutdown > Finish in-flight work before the process exits When a service is stopped, for example during a deploy or when `encore run` restarts it, it receives `SIGTERM` (or `SIGINT`, and Ctrl+C or Ctrl+Break on Windows). Encore.rs handles the signal for you: it stops taking new Pub/Sub messages, waits for in-flight message handlers, closes connections, flushes metrics and traces, and exits the process. Your HTTP server is part of your app, not Encore, so Encore can't drain its requests on its own. The `encore::shutdown` module lets the two cooperate: your server stops accepting new requests and finishes the ones in flight, and Encore cleans up after that. ## Shutting down an axum server Pass `encore::shutdown::signal()` to your server's graceful shutdown, then await `encore::shutdown::drained()` and exit with the code it returns: ```rust -- src/main.rs -- #[tokio::main] async fn main() { encore::initialize(); let app = axum::Router::new().route("/", axum::routing::get(|| async { "Hello!" })); let addr = std::env::var("ENCORE_LISTEN_ADDR").unwrap_or_else(|_| { let port = std::env::var("PORT").unwrap_or_else(|_| "8080".to_string()); format!("0.0.0.0:{port}") }); let listener = tokio::net::TcpListener::bind(&addr).await.unwrap(); axum::serve(listener, app) .with_graceful_shutdown(encore::shutdown::signal()) .await .expect("serve"); std::process::exit(encore::shutdown::drained().await.code()); } ``` The sequence on a shutdown signal is: 1. `signal()` resolves, and axum stops accepting connections and waits for in-flight requests to finish. 2. At the same time, Encore stops Pub/Sub subscriptions from fetching new messages and waits for the handlers already running. 3. When `serve` returns, `drained()` tells Encore your app has finished its work. 4. Encore closes its Postgres, Redis, and object storage connections, exports a final round of metrics, and sends any buffered trace spans. 5. `drained()` returns a `ShutdownResult`, and your app exits with its `code()`. Database and other infrastructure connections stay open until your app has drained, so requests that finish during shutdown can still use them. ## `signal()` ```rust pub fn signal() -> impl Future + Send + 'static ``` Returns a future that resolves once a shutdown signal is received. Calling it is what makes shutdown cooperative: from then on, Encore's cleanup waits for `drained()` (or the deadline) instead of starting immediately. Call it at startup, before the signal can arrive, as passing it to `with_graceful_shutdown` does. The future can be awaited anywhere, for example to stop a background loop: ```rust let stop = encore::shutdown::signal(); tokio::select! { _ = stop => {} _ = run_worker_loop() => {} } ``` ## `drained()` ```rust pub async fn drained() -> ShutdownResult ``` Tells Encore your app has finished its work, then waits for Encore's cleanup to complete. If no signal has started a shutdown yet, `drained()` starts one, so you can also call it when your app decides to exit on its own. It returns `encore::shutdown::ShutdownResult`: | Variant | Meaning | `code()` | |---|---|---| | `Success` | Cleanup finished within the deadline. | `0` | | `Timeout` | The deadline passed before cleanup finished. | `1` | | `Failed` | Cleanup stopped unexpectedly. | `1` | Exit with `result.code()` once `drained()` returns. Encore leaves exiting to your app in this case, so that code after `drained()` still runs. ## The deadline Shutdown has a deadline: five seconds unless the environment configures another. If your app hasn't drained by then, Encore proceeds with cleanup anyway and the result is `Timeout`. Buffered traces and metrics are still flushed after a timeout, for up to five more seconds. If your app called `signal()` but never calls `drained()`, Encore exits the process with code `1` once cleanup completes after the deadline. As a last resort, a process that is still running well after the deadline is forced to exit with code `1`. ## Apps that don't use `signal()` If you never call `signal()`, shutdown isn't cooperative. On a signal Encore drains Pub/Sub, closes connections, flushes telemetry, and exits the process with the result's code right away, without waiting for your HTTP server. In-flight requests are cut off, and requests still running may find their database connections closed. Use `signal()` and `drained()` in any service that serves requests. ## Services that serve no requests A service that only runs Pub/Sub subscriptions or other background work can call `encore::run_blocking()` from a synchronous `main`. It starts the registered subscriptions, blocks until a shutdown signal, drains, and exits the process: ```rust fn main() { encore::initialize(); encore::run_blocking(); } ``` `run_blocking()` blocks the calling thread on Encore's own runtime, so call it from a plain `fn main`, not from inside an async context such as `#[tokio::main]`. In an async `main`, await `signal()` and then `drained()` instead: ```rust #[tokio::main] async fn main() { encore::initialize(); encore::shutdown::signal().await; std::process::exit(encore::shutdown::drained().await.code()); } ``` ## Tests Encore doesn't install its signal handling under `encore test`. A test that awaits `signal()` waits forever. --- Source: https://encore.dev/docs/rs/develop/policies Section: Building (Encore.rs) # Policies > Rules about who may use what, checked against your code A policy is a rule about your app that Encore checks every time it reads your code. "Only services in `billing/payments` may write to databases holding card data." "A service may not call into another domain's services." "Every public endpoint must be exposed on the `api` gateway." You write the rule once, and Encore checks every service, resource and use of a resource against it, including the ones added after the rule was written. Policies are written in [CEL](https://cel.dev), a small expression language, over a model of your app that Encore builds from your code: its services, resources and endpoints, which services use which resources and how, and which [domains](#domains) and [tags](#tags) everything belongs to. They live in your `encore.app.json`, or in the `encore.domain.json` of a domain. > **Important** > > A broken policy never stops `encore run`. Your app keeps running, and Encore reports what its policies find next to it: in the terminal, in `encore policy check`, and to AI agents through the MCP server. ## An example A payments team keeps card data in a database tagged `data: pci`, and wants only its own services to write to it: ```json -- encore.app.json -- { "id": "shop", "policies": [ { "name": "pci-writes", "subjects": ["resource_link"], "match": "self.resource.effective_tags.data == 'pci' && self.access >= Access.WRITE", "require": "within('billing/payments', self.service.domain)", "message": "{self.service.name} writes PCI data in {self.resource.name}" } ] } ``` The policy applies to every `resource_link`, which is one service's use of one resource. `match` picks the uses it is about: writes to anything tagged `data: pci`. `require` says what each of those must make true: the service doing the writing is in the `billing/payments` domain. When a `reports` service, which is in no domain, starts writing to the `ledger` database, Encore reports: ``` error[ENC0061]: policy violation in `pci-writes`: reports writes PCI data in ledger --> reports/src/main.rs:6:5 | 6 | LEDGER.execute("update entries set amount = 0", &[]).await; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ reports writes ledger here | ::: encore.app.json:8:46 | 8 | "require": "within('billing/payments', self.service.domain)", | ------------------- no domain | ::: reports/encore.service.json:1:11 | 1 | { "name": "reports", "lang": "rust" } | --------- reports is in no domain ``` It points at the line of code that breaks the policy, underlines the part of `require` that came out false with its value, and shows where that value comes from. ## Domains A domain is a part of your app with a name: a directory with an `encore.domain.json` in it, and everything below that directory. Services, resources, frontends and gateways belong to the domain around the directory they are declared in. ```json -- billing/payments/encore.domain.json -- { "name": "payments", "description": "Card payments and the ledger.", "tags": { "data": "pci" } } ``` | Field | Required | Description | |---|---|---| | `name` | Yes | The domain's name, unique among its siblings. Lowercase letters, digits and `-`. | | `description` | No | What the domain is for. | | `tags` | No | The domain's [tags](#tags), which everything in it gets. | | `policies` | No | The domain's own [policies](#a-domain-s-policies). | Domains nest. A domain's id is its name, after the names of the domains it is in: `billing/payments` for a `payments` domain inside a `billing` domain. The id comes from the names, not the directory names, so the `payments` domain above could be in a directory called anything. Your app itself is the root domain, with the id `""`. Everything that is in no declared domain is in the root, so every service and resource always has a domain. A domain can't be declared inside a service's directory, though a service directory can be the root of one. Domains don't change how your app builds or runs: they exist for policies, and for Encore's tools to group your app by. ## Tags A tag is a key and a value, like `data: pci` or `team: payments`, that says something about a part of your app. Domains, services, frontends, gateways and resources can all have tags. A domain's tags go in its `encore.domain.json`, and a service's in its `encore.service.json`: ```json -- reports/encore.service.json -- { "name": "reports", "lang": "rust", "tags": { "team": "analytics" } } ``` A resource's tags go in its declaration: ```rust -- billing/payments/vault/src/db.rs -- use encore::storage::postgres::Database; pub static LEDGER: Database = Database::new("ledger").tag("data", "pci").tag("tier", "core"); ``` Every resource declaration takes `.tag(key, value)`, once per tag: databases, buckets, topics, subscriptions, Redis databases, cron jobs, secrets, configs, metrics and context fields. Keys and values must be string literals, because Encore reads them from the code without running it. Keys are lowercase letters, digits, `_` and `-`, start with a letter, and are at most 63 characters long. Values follow the same rules but may be empty. Something can have at most 32 tags, and keys starting with `encore` are reserved. Each part of your app has two sets of tags in the policy model: - `tags`: its own, as declared. - `effective_tags`: the tags of the domains it is in, merged from the outermost to the innermost, then its own. The nearer one wins when two set the same key. The `ledger` database above has `data: pci` among its effective tags because its domain sets it, without declaring it itself. Policies usually read `effective_tags`. A tag something doesn't have reads as the empty string, so `self.tags.data == 'pci'` is simply false for something with no `data` tag. To test whether a tag is set at all, use `has(self.tags.data)`. ## Writing a policy A policy is an object in the `policies` list of `encore.app.json` or of an `encore.domain.json`: | Field | Required | Description | |---|---|---| | `name` | Yes | Unique among the file's policies. Lowercase letters, digits and `-`, starting with a letter. | | `subjects` | Yes | The kinds of thing the policy judges. See [Subjects](#subjects). | | `match` | No | A CEL expression choosing which of those things the policy applies to. Without it, the policy applies to all of them. | | `require` | Yes | A CEL expression each thing the policy applies to must make true. | | `effect` | No | `"deny"`, the default, or `"warn"`. See [Effects](#effects). | | `message` | No | What a violation says. See [Messages](#messages). | Encore checks a policy when it reads it, before judging anything by it. A field the subject doesn't have, a misspelled function, comparing an enum with a string, or an expression that isn't true or false is reported as `ENC0060`, with a suggestion where there is one, and the policy is left out until it is fixed. ### Subjects A policy judges one or more kinds of subject, each in turn as `self`: | Subject | `self` is | |---|---| | `domain` | A domain, the root among them. | | `service` | A service. | | `frontend` | A frontend. | | `gateway` | A gateway. | | `resource` | A resource: a database, bucket, topic, subscription, Redis database, cron job, secret, config, metric or context field. | | `endpoint` | An endpoint of a service. | | `resource_link` | One service's use of one resource: every use in its code, together. | | `service_link` | One service calling another. | | `subscription_link` | One service subscribing to a topic. | | `exposure_link` | One endpoint exposed on one gateway: reachable from outside the app through it. | A policy can name several kinds of subject, like `["service", "resource"]`. Then `self` has only the fields all of them share, here `name`, `domain`, `tags`, `effective_tags` and `location`. Encore tells you which kinds have a field you use when not all of them do. ### Variables Every expression reads three variables: | Variable | What it is | |---|---| | `self` | The subject being judged. | | `app` | The whole app: `app.services`, `app.resources` and `app.domains` by name or id, `app.endpoints`, and every kind of link. | | `domain` | The domain the policy belongs to: the one whose `encore.domain.json` declares it, or the root for a policy in `encore.app.json`. | ### What a policy can read The most useful fields of each subject: | Subject | Fields | |---|---| | Domain | `id`, `name`, `parent`, `ancestry` (the ids of the domains it is in and its own, the root first), `depth`, `tags`, `effective_tags` | | Service | `name`, `language`, `domain`, `tags`, `effective_tags`, `root_dir`, `workers`, `gateways` (those any of its endpoints is exposed on), `serves_traffic`, `hosts_frontend`, `hosts_gateway` | | Frontend | `name`, `language`, `service` (the one hosting it, if any), `domain`, `tags`, `effective_tags` | | Gateway | `name`, `default`, `service` (the one running its middleware, if any), `domain`, `tags`, `effective_tags`, `origins_with_credentials`, `origins_without_credentials`, `any_origin_with_credentials` | | Resource | `id` (like `postgres_database:ledger`), `kind`, `name`, `domain`, `tags`, `effective_tags`, and its kind's own settings: `bucket.public`, `pubsub_topic.exactly_once`, `pubsub_subscription.max_retries`, `secret.scope`, and so on | | Endpoint | `service`, `name`, `protocol`, `methods`, `path`, `gateways`, `sensitive` | | Resource link | `service`, `resource`, `operations` (what the service does with it), `access` (the highest access those need), `locations`, `crossing` | | Service link | `caller`, `callee`, `locations`, `crossing` | | Subscription link | `service`, `subscription`, `topic`, `crossing` | | Exposure link | `gateway`, `endpoint`, `crossing` | Everything also has a `location`, with the `file`, `line` and `column` it is declared at. Every link has a `crossing`: the domain boundaries it crosses, from the side that acts to the side acted on. For a service in `billing/payments` using a resource in `identity`, `crossing.common` is the root, the innermost domain holding both; `crossing.exits` is `billing/payments` then `billing`; and `crossing.enters` is `identity`. A link inside one domain crosses nothing, so `self.crossing.enters.size() == 0`. Enumerations are written by their type and name: | Enumeration | Values | |---|---| | `Access` | `READ`, `WRITE`, `ADMIN`, in that order, so `self.access >= Access.WRITE` holds for writes and schema changes. | | `Operation` | `POSTGRES_READ`, `POSTGRES_WRITE`, `POSTGRES_ADMIN`, `BUCKET_LIST`, `BUCKET_READ`, `BUCKET_WRITE`, `BUCKET_DELETE`, `BUCKET_SIGNED_UPLOAD_URL`, `TOPIC_PUBLISH`, `SUBSCRIPTION_RECEIVE`, `REDIS_READ`, `REDIS_WRITE`, `SECRET_READ`, `CONFIG_READ`, `METRIC_RECORD`, and more. | | `Resource.Kind` | `POSTGRES_DATABASE`, `BUCKET`, `PUBSUB_TOPIC`, `PUBSUB_SUBSCRIPTION`, `REDIS_DATABASE`, `CRON_JOB`, `SECRET`, `CONFIG`, `METRIC`, `CONTEXT_FIELD` | | `Resource.Scope` | `SERVICE`, `SHARED`, `GLOBAL` | | `Service.Language` | `GO`, `TYPESCRIPT`, `RUST`, `PYTHON` | | `Endpoint.Protocol` | `HTTP`, `GRAPHQL`, `GRPC`, `PASSTHROUGH`, `FRAMEWORK` | `encore policy eval` [shows](#trying-an-expression) any field's value for any subject. ### Functions Policies can use CEL's standard functions, like `size()`, `startsWith()`, `matches()`, `in`, `all()` and `exists()`, and its strings extension. Encore adds three: | Function | What it does | |---|---| | `within(id, d)` | Whether the domain `d` is the domain `id` or is inside it. `within('billing', self.service.domain)` holds for a service in `billing` or `billing/payments`. | | `glob(pattern, s)` | Whether `s` matches `pattern`, where `*` matches any run of characters and `?` any one. `glob('internal-*', self.name)`. | | `reaches(s)` | The resources the service `s` can reach: those it uses, and those reached by the services it calls and by the subscribers of the topics it publishes to. | ### Effects A policy with `"effect": "deny"`, the default, reports what breaks it as an error. `encore policy check` exits with an error when there is one, so a CI step can stop a change that breaks a policy. A policy with `"effect": "warn"` reports what breaks it as a warning, and stops nothing. It is the way to introduce a rule to an app that doesn't keep it yet. ### Messages A violation's message is the policy's `message`, with each `{expression}` in it replaced by the expression's value for the subject that broke it: ```json "message": "{self.service.name} writes PCI data in {self.resource.name}" ``` Strings are written as they are, enum values by their names (`WRITE`), and lists joined by `, `. Write `{{` and `}}` for literal braces. Without a `message`, a violation says what broke the policy, like `reports using database ledger`. ## A domain's policies A domain can declare policies of its own, in its `encore.domain.json`. They judge only what is in the domain, and the links with an end in it: a domain's policy over `service_link` sees calls into the domain from outside as well as calls out of it. In its expressions, `domain` is the domain itself, so the same policy reads the same in any domain: ```json -- billing/encore.domain.json -- { "name": "billing", "policies": [ { "name": "no-calls-out", "subjects": ["service_link"], "match": "within(domain.id, self.caller.domain)", "require": "within(domain.id, self.callee.domain)", "effect": "warn", "message": "{self.caller.name} calls {self.callee.name}, outside {domain.id}" } ] } ``` The app's own policies, in `encore.app.json`, judge everything. ## More examples Only the `api` gateway exposes endpoints to the internet: ```json { "name": "one-gateway", "subjects": ["exposure_link"], "require": "self.gateway.name == 'api'" } ``` No public buckets, outside the `marketing` domain: ```json { "name": "private-buckets", "subjects": ["resource"], "match": "self.kind == Resource.Kind.BUCKET && !within('marketing', self.domain)", "require": "!self.bucket.public" } ``` Every service says which team owns it: ```json { "name": "owned", "subjects": ["service"], "require": "has(self.effective_tags.team)", "effect": "warn", "message": "{self.name} has no team tag" } ``` A service that handles personal data can't reach a database tagged `analytics`: ```json { "name": "pii-stays-out-of-analytics", "subjects": ["service"], "match": "self.effective_tags.data == 'pii'", "require": "!reaches(self).exists(r, r.effective_tags.data == 'analytics')" } ``` Only schema migrations change a database's schema, so no service code needs admin access: ```json { "name": "no-admin", "subjects": ["resource_link"], "require": "self.access < Access.ADMIN" } ``` ## Checking your app `encore run` checks your app against its policies each time it reads your code, and shows what they find under the app's output. In the terminal UI, press `p` to see every finding. With `--json`, each report is an event of kind `policy`. `encore policy check` checks the app once, prints each finding, and exits with an error if a `deny` policy is broken or a policy can't be read: ```shell $ encore policy check ``` `encore policy ls` lists the app's policies, where each is declared, and how many things break it. When a policy can't be decided for something, because an expression fails on it, the policy counts as broken by it. The usual cause is reading a key a map doesn't have, like `app.services['billing']` in an app with no `billing` service: test for it first, with `has()` or `in`. ### Trying an expression `encore policy eval` evaluates one expression, the way a policy's `match` or `require` would, and prints its value: ```shell $ encore policy eval 'self.access' --subject 'resource_link:reports->postgres_database:ledger' "WRITE" (Access) ``` The subject is named the way `encore policy check` names it: `service:billing`, `resource:postgres_database:ledger`, `domain:billing/payments`, `service_link:reports->billing`. Without `--subject`, the expression can read only `app` and `domain`. Use `--domain` to evaluate it as a policy of that domain would be. ### For AI agents Encore's [MCP server](https://encore.dev/docs/mcp) has the tools `policy_check`, `policy_list` and `policy_eval`, which do what the commands above do. Its other tools, like the ones that call an endpoint or run tests, add a warning to their output while a policy is broken, so an agent finds out it broke one without being stopped. --- Source: https://encore.dev/docs/platform/introduction Section: Deploying # Encore Platform > The managed platform that provisions and operates AWS and GCP from your application code Cloud platforms are powerful, but the path from code to a running production system on AWS or GCP usually involves Terraform, YAML, container orchestration, IAM policies, secret managers, and a CI/CD pipeline to glue it all together. Most teams end up building (and maintaining) their own internal developer platform on top of all that. The Encore platform removes the need for that internal platform. It pairs with the open source [Encore.ts](https://encore.dev/docs/ts) and [Encore.go](https://encore.dev/docs/go) infrastructure SDKs to take an application from local development to a production deployment in your own cloud, with the SDK declarations as the single source of truth. ## How the Encore platform works with the SDKs The infrastructure SDKs let you declare the resources your application needs (SQL databases, Pub/Sub topics, object storage, caches, cron jobs, secrets) directly in your TypeScript or Go code, as typed objects. The Encore platform reads those declarations to build an infrastructure model of your application. It then uses that model to: 1. Provision matching resources in your AWS or GCP account using the cloud provider's API. 2. Wire the running application up to those resources at deploy time (connection strings, IAM, secret injection). 3. Keep the model and the running infrastructure in sync as your code changes. The resources used in production are exactly the ones your code asked for, nothing more, nothing less. ![How the Encore platform provisions infrastructure from your code](https://encore.dev/assets/docs/howitworks.png) When your application is deployed, **there are no runtime dependencies on the Encore platform** and **no proprietary code runs in your cloud**. The application is a standard service talking to standard managed AWS or GCP resources. ## A tight iteration loop, local to production Because the SDK is the source of truth for your infrastructure, the same model runs locally, in per-PR preview environments, and in production: 1. **Local:** `encore run` boots the whole system on your laptop with real Postgres, real Pub/Sub semantics, and real tracing. [Infrastructure namespaces](https://encore.dev/docs/infra-namespaces) let multiple branches or agents work in parallel with isolated local state. 2. **Per-PR preview environments:** On paid plans, each pull request gets an ephemeral [preview environment](https://encore.dev/docs/platform/deploy/preview-environments), optionally with a [database branched from a seed environment](https://encore.dev/docs/platform/infrastructure/neon). End-to-end validation against real cloud services before merge. 3. **Production:** The same declarations provision the production resources on deploy. This loop is what's structurally missing from Terraform-based workflows, which tend to be production-centric and hard to run on a laptop. The tight feedback loop is especially impactful for [AI coding agents](https://encore.dev/docs/platform/ai-integration), which work best when they can validate their own changes against real infrastructure rather than guess. See the [Development Workflow](https://encore.dev/docs/platform/workflow) page for the full picture. ## What the Encore platform gives you ### Infrastructure provisioning in your own cloud - **Direct provisioning via cloud APIs:** The Encore platform creates resources using your cloud provider's official APIs. No IaC files to maintain; your code is the source of truth. See [Provisioning & Environments](https://encore.dev/docs/platform/infrastructure/infra). - **Battle-tested managed services:** Provisions Cloud Run / Fargate, GKE / EKS, CloudSQL / RDS, Pub/Sub / SNS+SQS, and so on, with sensible defaults per cloud. - **Least-privilege IAM, generated from code:** Permissions are derived from how each resource is used, not hand-written policies. - **Infrastructure inventory and change management:** See every provisioned resource and review infrastructure changes before they're applied. ### Deployments and environments - **Zero-config deploys:** Connect a [GitHub repo](https://encore.dev/docs/platform/integrations/github) and your cloud account; push to deploy. See [Deploying & CI/CD](https://encore.dev/docs/platform/deploy/deploying). - **[Preview environments](https://encore.dev/docs/platform/deploy/preview-environments) per pull request:** Real cloud resources, automatically created and torn down. - **Multiple environments out of the box:** Staging, production, and ad-hoc environments share the same code path. See [Environments](https://encore.dev/docs/platform/deploy/environments). - **Secrets management:** Reference secrets by name in code; Encore stores them in your cloud's secret manager and injects them at runtime. ### Observability - **Distributed tracing, metrics, and logs**, with no agent code in your application. See [Tracing](https://encore.dev/docs/platform/observability/tracing) and [Metrics](https://encore.dev/docs/platform/observability/metrics). - **Third-party export:** Send telemetry to Datadog, Grafana, or other tools you already use. - **Cost monitoring** across your cloud resources (GCP today, AWS in progress). ### Documentation and discovery The infrastructure model also drives a [Service Catalog](https://encore.dev/docs/service-catalog) with API docs and a live [architecture diagram](https://encore.dev/docs/architecture). Useful for onboarding new engineers and for AI tooling, but not the reason most teams adopt Encore. ## You're not locked in The Encore platform is optional. The SDKs are open source and produce a standard application binary or Node.js process. If you want to run things yourself: - Build a Docker image with `encore build docker` and deploy it anywhere. - Provision your own infrastructure with [Terraform](https://encore.dev/docs/platform/integrations/terraform), Pulumi, CloudFormation, or the cloud console. - Tell the Encore runtime how to reach your resources via an [infrastructure config file](https://encore.dev/docs/self-host/configure-infra). See the [self-hosting guide](https://encore.dev/docs/self-host/build) for the full workflow. ## When the Encore platform is a good fit The Encore platform suits teams that want their application's infrastructure to be defined and operated from the same codebase as the application itself, deployed into AWS or GCP accounts they own. Typical use cases include: - Consumer apps and B2B platforms - Fintech and crypto applications - E-commerce marketplaces - Microservices backends and event-driven systems See the [showcase](https://encore.dev/showcase) for production examples, or read [customer stories](https://encore.dev/customers) for how specific teams use it. ## Getting started - Start with the [Quick Start for TypeScript](https://encore.dev/docs/quick-start) or [Go](https://encore.dev/docs/quick-start). - [Connect your AWS or GCP account](https://encore.dev/docs/platform/deploy/own-cloud). - [Book a demo](https://encore.dev/book) to talk through whether the Encore platform fits your team. - [Join Discord](https://encore.dev/discord) to ask questions and meet other Encore developers. --- Source: https://encore.dev/docs/platform/workflow Section: Deploying # Development Workflow > Declare once, then run and validate the same application at every stage An Encore change follows one path from a developer's laptop to production: declare what the application needs, run it locally, test it in isolation, review it in a preview environment, and deploy it. The resource declarations do not change between those stages. A database declaration can become Postgres in Docker locally, an isolated test database during `encore test`, and RDS or Cloud SQL in production. Each environment supplies a different implementation of the same logical resource. ## Declare what the application needs Encore's infrastructure primitives are SDK types for common backend resources: SQL databases, Pub/Sub, object storage, caches, cron jobs, and secrets. You declare them in the application code that uses them. Services and APIs are defined in code as well. These declarations describe the application's logical requirements, not a particular cloud setup. A database declaration gives the resource a name and a migrations directory; it does not choose an instance size or database engine. Those physical properties belong to the environment's [infrastructure configuration](https://encore.dev/docs/platform/infrastructure/configuration). Encore uses static analysis to connect each declaration with the services that use it. The resulting [application model](https://encore.dev/docs/understanding-encore) drives local infrastructure, validation, service discovery, tracing, IAM, and cloud provisioning. See the [Encore.ts](https://encore.dev/docs/ts) and [Encore.go](https://encore.dev/docs/go) overviews for the complete set of primitives and code examples. [Infrastructure on AWS and GCP](https://encore.dev/docs/platform/infrastructure/infra) shows what each primitive becomes in every environment. ## Run locally `encore run` starts the application and the local infrastructure its declarations require. That includes Postgres in Docker, an in-memory Pub/Sub broker and cache, and object storage on the local filesystem. There is no separate Docker Compose file or emulator configuration to keep in sync. It also starts the [local development dashboard](https://encore.dev/docs/dev-dash), where you can call APIs and inspect the application's services, databases, logs, and distributed traces. Cron jobs are not triggered automatically in local or preview environments. You can invoke their endpoints from the dashboard. When several developers or coding agents work on the same application, [infrastructure namespaces](https://encore.dev/docs/infra-namespaces) isolate their local state. Each namespace has its own databases and other resources, and switching back restores the state you left behind: ```shell encore namespace switch --create pr:123 ``` ## Test in isolation `encore test` starts infrastructure in test mode, then hands control to Vitest or Jest for TypeScript, or `go test` for Go. Each run gets isolated databases tuned for test speed, and object storage runs in memory. Tests can exercise database queries, Pub/Sub handlers, and calls between services without separate test infrastructure. See automated testing for [TypeScript](https://encore.dev/docs/ts/develop/testing) or [Go](https://encore.dev/docs/go/develop/testing). ## Review in a preview environment For applications connected to GitHub, Encore can create a [preview environment](https://encore.dev/docs/platform/deploy/preview-environments) for each pull request. It deploys the proposed code with the same resource declarations as production, giving the change its own URL, infrastructure, logs, and traces. Preview environments run on Encore-managed infrastructure by default. Enterprise teams can instead host them in their own cloud. You can also [branch a database from a seed environment](https://encore.dev/docs/platform/infrastructure/neon) when a preview needs representative data. This is the first stage that exercises the change against cloud infrastructure, while keeping it isolated from production and other pull requests. ## Deploy to production Deploying reads the same application model that was validated locally and in preview. Encore provisions the matching resources in your AWS or GCP account and deploys the application services that use them. Adding a declaration adds the corresponding cloud resource on the next deploy. Environment-specific settings such as compute, capacity, networking, backups, and database engines remain outside the application code. You manage them through Encore while retaining access to the resources in your cloud provider's console. Because the application model records resource usage and service calls, Encore can derive [least-privilege IAM policies](https://encore.dev/docs/platform/deploy/security) for the deployed application. ## Why the loop works well for coding agents Coding agents are most useful when they can check their own work. Encore gives an agent feedback at every stage: 1. The compiler reports invalid declarations and resource usage. 2. `encore run` provides the application and its infrastructure for local inspection. 3. `encore test` runs the change against isolated test infrastructure. 4. A preview environment validates it against cloud services before merge. 5. Production uses the declarations already exercised in the earlier stages. [AI Integration](https://encore.dev/docs/ai-integration) covers the rules and MCP server that give an agent access to services, schemas, and traces. [AI infrastructure provisioning](https://encore.dev/docs/platform/ai-integration) covers the controls on infrastructure changes. ## Where to go next - Build and run an application with the [Quickstart guide](https://encore.dev/docs/quick-start). - Explore the infrastructure primitives in [Encore.ts](https://encore.dev/docs/ts) or [Encore.go](https://encore.dev/docs/go). - Learn how the declarations fit together in [Understanding Encore](https://encore.dev/docs/understanding-encore). - Configure [preview environments](https://encore.dev/docs/platform/deploy/preview-environments) and [deployments](https://encore.dev/docs/platform/deploy/deploying). --- Source: https://encore.dev/docs/platform/ai-integration Section: Deploying # AI Integration > AI agents that can provision real infrastructure in your cloud The Encore platform supercharges AI-powered development by letting AI agents provision real infrastructure in your AWS or GCP account with automatic guardrails. When you connect your cloud account to [the Encore platform](https://encore.dev/docs/platform/introduction), AI-generated code that declares databases, pub/sub topics, cron jobs, and other [primitives](https://encore.dev/docs/ts) gets automatically provisioned with production-ready defaults: proper networking, IAM permissions, and security configurations. ## How It Works 1. **AI writes infrastructure code** using Encore's declarative primitives 2. **Push to GitHub** to trigger a deployment 3. **Encore platform provisions** the infrastructure in your AWS/GCP account with automatic guardrails 4. **Preview environments** let you test AI-generated changes in isolation This enables fast iterative development: AI generates code, you push it and validate in preview environments, then deploy seamlessly with automatic infrastructure provisioning. ## Infrastructure with Guardrails When AI declares infrastructure using Encore primitives, the Encore platform provisions it in your cloud with automatic guardrails: - **Databases**: Proper networking, encryption at rest, automated backups - **Pub/Sub**: Dead letter queues, retry policies, proper IAM roles - **Secrets**: Encrypted storage, access controls - **Services**: Load balancing, health checks, auto-scaling AI doesn't need to know the intricacies of AWS or GCP. It just declares what it needs, and the Encore platform handles the cloud-specific configuration. You stay in control: review infrastructure changes in pull requests, approve or deny resource additions, and use [infrastructure configuration](https://encore.dev/docs/platform/infrastructure/configuration) to customize defaults per environment. ## Preview Environments [Preview environments](https://encore.dev/docs/platform/deploy/preview-environments) are perfect for testing AI-generated changes. Each pull request gets its own isolated environment with real infrastructure. This means you can: - Let AI generate features and immediately test them with real databases and services - Review AI-generated infrastructure changes before they hit production - Catch issues early in isolated environments ## Production Observability The Encore platform provides [distributed tracing](https://encore.dev/docs/platform/observability/tracing) and [metrics](https://encore.dev/docs/platform/observability/metrics) across all your environments. You can: - Analyze traces to debug issues across services - Inspect timing to find bottlenecks - Compare behavior between preview and production environments ## MCP Servers Encore provides [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) servers that give AI agents rich context about your application. MCP is an open standard that lets large language models access contextual information from external sources. ### Local MCP Server The [Local MCP Server](https://encore.dev/docs/mcp) runs alongside `encore run` and gives AI tools access to your local application's metadata, traces, databases, source code, and more. It's ideal for connecting AI assistants like Cursor or Claude Desktop to your local development environment. ### Cloud MCP Server The [Cloud MCP Server](https://encore.dev/docs/platform/cloud-mcp) gives AI agents access to your deployed environments on Encore Cloud, including production traces, deployment status, environment configuration, and infrastructure metadata. It's hosted at `https://api.encore.dev/mcp` and authenticates via OAuth or a Bearer API token. ## Connecting Your Cloud To deploy your Encore app to your cloud: 1. [Sign up for the Encore platform](https://app.encore.dev) 2. [Connect your AWS or GCP account](https://encore.dev/docs/platform/deploy/own-cloud) 3. Push your Encore app to deploy The Encore platform provisions infrastructure in your cloud account based on the primitives declared in your code. You maintain full control and ownership of your infrastructure. ## What AI Can Provision With the Encore platform connected, AI-generated code can provision: | Resource | AWS | GCP | |----------|-----|-----| | Databases | RDS (PostgreSQL) | Cloud SQL | | Pub/Sub | SNS + SQS | Cloud Pub/Sub | | Object Storage | S3 | Cloud Storage | | Cron Jobs | CloudWatch Events | Cloud Scheduler | | Secrets | Secrets Manager | Secret Manager | | Caching | ElastiCache | Memorystore | All resources are provisioned with security best practices like least-privilege IAM policies, private networking, and encryption. See the [AWS](https://encore.dev/docs/platform/infrastructure/aws) and [GCP](https://encore.dev/docs/platform/infrastructure/gcp) infrastructure docs for specifics. ## Learn More - [Connect Your Cloud Account](https://encore.dev/docs/platform/deploy/own-cloud) - [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments) - [Infrastructure Configuration](https://encore.dev/docs/platform/infrastructure/configuration) - [Framework AI Integration (TypeScript)](https://encore.dev/docs/ai-integration) - [Framework AI Integration (Go)](https://encore.dev/docs/ai-integration) --- Source: https://encore.dev/docs/platform/cloud-mcp Section: Deploying # Cloud MCP Server > Give AI agents access to your deployed environments on Encore Cloud The Encore platform MCP Server gives AI agents access to your deployed environments on Encore Cloud, including production traces, deployment status, environment configuration, and infrastructure metadata. It implements the [Model Context Protocol](https://modelcontextprotocol.io/introduction), an open standard for connecting large language models to external context sources. For AI access to your local development environment, see the [Local MCP Server](https://encore.dev/docs/mcp). ## Endpoint Connect your AI tool to the Cloud MCP Server at: ``` https://api.encore.dev/mcp ``` ## Authentication The Cloud MCP Server supports two authentication methods: - **OAuth** — interactive flow that opens a browser to grant access. Best for local development. - **API Key** — Bearer token sent as an `Authorization` header. Best for CI environments or shared configuration. ## Connect Claude Code ### With OAuth Add the Cloud MCP Server to Claude Code: ```bash claude mcp add --transport http encore-cloud https://api.encore.dev/mcp ``` The first time you use the server, Claude Code will open a browser window to complete the OAuth flow and grant access to your Encore account. Verify with `claude mcp list`. You should see `encore-cloud` in the list. ### With an API Key If you'd rather not use OAuth (for example, in CI environments or when sharing configuration), you can authenticate with a Bearer API token instead. **1. Generate an API key** In the [Encore platform dashboard](https://app.encore.dev), navigate to **Settings → API Keys** for your app and generate a new API key. Copy the token — you won't be able to see it again. **2. Configure Claude Code with the API key** Add the Cloud MCP Server with an `Authorization: Bearer` header: ```bash claude mcp add --transport http encore-cloud https://api.encore.dev/mcp \ --header "Authorization: Bearer " ``` When authenticating with an API key, Claude Code skips the OAuth flow and uses the bearer token for every request. Treat the token like a password and avoid committing it to source control. --- Source: https://encore.dev/docs/platform/deploy/deploying Section: Deploying # Deploying Applications with Encore > Deploy with a git push, while Encore provisions your infrastructure ## Deploying your first app Automated deploys need a free Encore account: you push to a git remote, and the platform builds, provisions and deploys for you. Prefer to manage the infrastructure yourself? [Build a Docker image](https://encore.dev/docs/self-host/build) instead. ### Starting from scratch Create your Encore account, then create the app: ```shell $ encore auth signup $ encore app create ``` `encore app create` scaffolds a new app, registers it to your account, and adds the `encore` git remote. Nothing else to set up: skip ahead to [Push to deploy](#push-to-deploy). ### Created your app in the Cloud Dashboard? Apps created from a template in the Cloud Dashboard are deployed once automatically. To change the code and deploy again, install the Encore CLI: - **macOS:** `brew install encoredev/tap/encore` - **Linux:** `curl -L https://encore.dev/install.sh | bash` - **Windows:** `iwr https://encore.dev/install.ps1 | iex` Then clone the app. Your app ID is in the dashboard URL: `app.encore.dev/`. ```shell $ encore auth login $ encore app clone $ cd $ encore run # Optional. Apps with a database need Docker running. ``` `encore app clone` creates a directory named after the app ID, with a git remote named `encore`. Make a change, then [push to deploy](#push-to-deploy). If you connected GitHub when you created the app, clone the GitHub repository instead, and push your changes there. ### Already have a local app? If you built your app before signing up, or created it from an example or starter template, link it to your account with the steps below. Run them from your app's root folder. **1. Sign up, or log in** ```shell $ encore auth signup # No account yet $ encore auth login # Already have one ``` **2. Remove the existing `encore.app` file** Examples and starter templates ship with one, and `encore app init` won't run while it's there. If it has settings other than `id` and `lang` (such as `build` or `global_cors`), copy them first so you can add them back after the next step: ```shell $ rm encore.app ``` **3. Register the app** Use `--lang ts` for an Encore.ts app, or `--lang go` for Encore.go: ```shell $ encore app init my-app --lang ts ``` This prints the app's new id, writes a fresh `encore.app` containing it, and adds the `encore` git remote, creating a git repository first if needed. (Without `--lang` the command still succeeds, but omits the `lang` field.) **4. Commit and push** ```shell $ git add -A . $ git commit -m 'Link to Encore Cloud' $ git push encore ``` If git reports that `encore` doesn't appear to be a git repository, add the remote with the id from the previous step: `git remote add encore encore://`. ## Integrate with GitHub (optional) New apps get an Encore managed git repository, which is enough to try things out. For production we recommend [integrating with GitHub](https://encore.dev/docs/platform/integrations/github) instead. Once connected, pushing to GitHub triggers deployments automatically, and Pro users get [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments) for each pull request. ## Connect your AWS or GCP account (optional) To deploy to your own cloud account rather than Encore Cloud hosting, see [connect your cloud account](https://encore.dev/docs/platform/deploy/own-cloud). Skipping this deploys to a free Encore hosted environment (Encore Cloud), subject to [fair use limits](https://encore.dev/docs/platform/management/usage). ## Push to deploy Deploy your application by pushing your code to the connected Git repository. - **Using Encore's managed git**: ```shell $ git add -A . $ git commit -m 'Commit message' $ git push encore ``` - **If you have connected your GitHub account:** ```shell $ git add -A . $ git commit -m 'Commit message' $ git push origin ``` This triggers the deployment process, consisting of the following phases: * A build & test phase * An infrastructure provisioning phase * A deployment phase Once you've pushed your code, you can monitor the progress in the **[Cloud Dashboard](https://app.encore.dev/) > (Select your app) > Deployments**. Every push to your app's main branch deploys the same way. When you push to `encore`, a successful push prints a line like this: ```shell remote: main: triggered deploy https://app.encore.dev//envs/staging/deploys/... ``` If you don't see it, nothing was deployed. Usually there was nothing new to push, so commit your changes first, or you pushed a branch that no environment deploys from. To deploy again without changing your code, for example after setting a missing secret, open **Deployments** in the Cloud Dashboard, click **New Deploy**, choose **Branch**, and select the branch the environment deploys from (usually `main`). (**Redeploy Latest** also works once the environment has a successful deploy.) If a deploy fails, see [Fixing a failed deploy](https://encore.dev/docs/platform/deploy/troubleshooting). ## Configuring deploy trigger When using GitHub, you can automatically trigger deploys when you push to a specific branch name. To configure which branch name is used to trigger deploys, open your app in the [Cloud Dashboard](https://app.encore.dev) and go to the **Overview** page for your intended environment. Click on **Settings** and then in the section **Branch Push** configure the `Branch name` and hit save. ### Integrating using the Encore API You can trigger deployments using Encore's API, learn more in the [API reference](https://encore.dev/docs/platform/integrations/api-reference). ## Configuring custom build settings If you want, you can override certain aspects of the CI/CD process in the `encore.app` file: * The Docker base image to use when deploying * Whether to build with Cgo enabled * Whether to bundle the source code in the docker image (useful for [Sentry stack traces](https://docs.sentry.io/platforms/go/usage/serverless/)) To install private npm packages or Go modules during the build, see [Private Packages](https://encore.dev/docs/platform/deploy/private-packages). Below are the available build settings configurable in the `encore.app` file, with their default values: ```cue -- encore.app -- { "build": { // Enables cgo when building the application and running tests // in Encore's CI/CD system. "cgo_enabled": false, // Docker-related configuration "docker": { // The Docker base image to use when deploying the application. // It must be a publicly accessible image. It defaults to "scratch" for go apps // and "node:24-trixie" for typescript apps. "base_image": "scratch", // Whether to bundle the source code in the docker image. // The source code will be copied into /workspace as part // of the build process. This is primarily useful for tools like // Sentry that need access to the source code to generate stack traces. "bundle_source": false, // The working directory to start the docker image in. // If empty it defaults to "/workspace" if the source code is bundled, and to "/" otherwise. "working_dir": "" } // Build hooks allow you to run custom commands during the build process. // They can be specified as a string (e.g. "cmd1 && cmd2") or as an object // with a command and optional environment variables. "hooks": { // Runs before the Encore build, but after dependencies are fetched (e.g. npm install). "prebuild": "", // Or as an object: // "prebuild": {"command": "my-command", "env": {"MY_VAR": "value"}}, // Runs after the Encore build has finished. "postbuild": "" // Or as an object: // "postbuild": {"command": "my-command", "env": {"MY_VAR": "value"}} } } } ``` --- Source: https://encore.dev/docs/platform/deploy/troubleshooting Section: Deploying # Fixing a failed deploy > Common errors when deploying with Encore, and how to fix them When a deploy fails, open it in the [Cloud Dashboard](https://app.encore.dev) under **Deployments** to see which step failed and the error it printed. These are the most common causes. ## Tests failed A build whose tests fail is not deployed. Run the tests locally to see the failure: ```shell $ encore test ``` Fix the test and push again. To deploy without running tests, either turn off **Run tests in every build** under **Settings** > **Builds** > **Tests**, or check **Skip Tests** when you start a deploy from **Deployments** > **New Deploy**. For Encore.ts apps, tests only run in the build if `package.json` has a `test` script. ## Missing secrets The deploy fails with: ``` secret key(s) not defined: MySecret ``` Every secret your app declares needs a value in the environment you deploy to. Set it for that environment's type: `dev` for development environments, such as the default `staging`, and `prod` for production. Or set it for one environment with `--env`: ```shell $ encore secret set --type dev MySecret $ encore secret set --env MySecret ``` You can also set secrets in the Cloud Dashboard under **Settings** > **Secrets**. Then redeploy without a new commit: open **Deployments**, click **New Deploy**, choose **Branch**, and select the branch the environment deploys from (usually `main`). See Secrets for [Encore.ts](https://encore.dev/docs/ts/primitives/secrets) or [Encore.go](https://encore.dev/docs/go/primitives/secrets) for more. ## `encore.app` isn't in the repository root The build fails with: ``` encore.app not found in the repository root ``` If your app lives in a sub-folder of the repository, set its path under **Settings** > **General** > **Root Directory**. ## The push didn't start a deploy When you push to the `encore` remote, a successful push prints a line like this: ``` remote: main: triggered deploy https://app.encore.dev//envs/staging/deploys/... ``` If you don't see it, nothing was deployed: - **`Everything up-to-date`:** there was nothing new to push. Commit your changes with `git add -A . && git commit -m '...'`, then push again. - **No `triggered deploy` line:** you pushed a branch that no environment deploys from. Push your app's main branch, or configure which branch an environment deploys from under the environment's **Settings** > **Branch Push**. ## `git push encore` fails - **`could not get Encore auth token: not logged in`:** run `encore auth login`, then push again. - **`'encore' does not appear to be a git repository`:** the `encore` remote is missing. Add it with `git remote add encore encore://`. Your app ID is in the dashboard URL: `app.encore.dev/`. - **The push hangs or times out:** `git push encore` and `encore app clone` connect to `git.encore.dev` over SSH on port 22. Some networks block outbound port 22. Try another network, or [connect GitHub](https://encore.dev/docs/platform/integrations/github) and push there instead. --- Source: https://encore.dev/docs/platform/deploy/own-cloud Section: Deploying # Connect your cloud account > Whatever cloud you prefer is fine by us The Encore platform lets you deploy your application to any of the major cloud providers, using your own cloud account. This lets you use Encore to improve your experience and productivity, while keeping the reliability of a major cloud provider. Each [environment](https://encore.dev/docs/platform/deploy/environments) can be configured to use a different cloud provider, and you can have as many environments as you wish. This also lets you easily deploy a hybrid or multi-cloud application, as you see fit. > **Note** > > The Encore platform will provision infrastructure in your cloud account, but for safety reasons the Encore platform does not automatically destroy infrastructure once it's no longer required. To do this, you need to manually approve the deletion of the infrastructure in your Encore dashboard. > > This means if you disconnect your app from your cloud provider, or delete the environment > within Encore, you need to explicitly approve the deletion of the infrastructure in your Encore dashboard. ## Google Cloud Platform (GCP) The Encore platform provides a GCP Service Account for each Encore application, letting you grant the Encore platform access to provision all the necessary infrastructure directly in your own GCP account. ### Permissions scoping GCP's permissions system is well-suited for scoping down the Encore platform's access. While the simplest setup grants access at the organization level, permissions can also be scoped down to a single GCP project. This is useful when you want to isolate the Encore platform's access to a specific project within your organization, for example a sandboxed prototyping environment. [Contact us](https://encore.dev/book) to discuss the best setup for your needs. ### Required permissions When connecting a specific GCP project (instead of granting access at the organization level), the simplest approach is to grant the Encore platform service account the `roles/owner` (Owner) role on the project. This gives the Encore platform full access to provision and manage all the infrastructure it needs. If you'd prefer to grant more narrowly scoped permissions, only grant the roles for the features your application actually uses. The following project-level roles are grouped by the feature they enable: #### IAM (always required) - `roles/resourcemanager.projectIamAdmin` — read/set project IAM policy - `roles/iam.serviceAccountAdmin` — create/update/delete service accounts - `roles/iam.roleAdmin` — create/patch/undelete the `encore_bucket_*` custom roles - `roles/iam.serviceAccountTokenCreator` — needed for Pub/Sub push-OIDC + workload identity flows - `roles/iam.serviceAccountUser` — needed to deploy Cloud Run services with a custom service account #### Service usage (always required) - `roles/serviceusage.serviceUsageAdmin` — enables the required `.googleapis.com` APIs #### Cloud Run (required if deploying to Cloud Run) - `roles/run.admin` — create/replace/delete Cloud Run services and their IAM policy - `roles/vpcaccess.admin` — manage serverless VPC connectors #### GKE (required if deploying to GKE) - `roles/container.admin` — clusters, node pools, labels #### Networking (required for ingress, custom domains, and private connectivity) - `roles/compute.networkAdmin` — VPC, subnets, firewalls, peering, reserved IPs, PSC forwarding rules - `roles/compute.loadBalancerAdmin` (or the broader `roles/compute.admin`) — URL maps, target proxies, forwarding rules, backend buckets/services - `roles/servicenetworking.networksAdmin` — private services access for Cloud SQL / Redis - `roles/certificatemanager.editor` — certificates, cert maps and entries, DNS authorizations #### Databases (required if using SQL databases or caches) - `roles/cloudsql.admin` — instances, users, SSL certs - `roles/redis.admin` — Memorystore instances #### Storage (required if using Object Storage) - `roles/storage.admin` — buckets, CORS, lifecycle, IAM #### Storage / Registry (always required) - `roles/artifactregistry.admin` — repositories for container images #### Messaging (required if using Pub/Sub topics) - `roles/pubsub.admin` — topics, subscriptions, and their IAM bindings #### Secrets (always required) - `roles/secretmanager.admin` — create/version/delete secrets and grant accessor #### Observability (always required) - `roles/monitoring.editor` — create/manage custom metric descriptors ### Setup To find your app's Service Account email and configure GCP deployments, head over to the Connect Cloud page by going to the **[the Encore platform dashboard](https://app.encore.dev/) > (Select your app) > App Settings > Integrations > Connect Cloud**. ![Connect GCP account](https://encore.dev/assets/docs/connectgcp.png "Connect GCP account") #### Encore Service Account vs Service Account JSON Key When connecting your GCP account, you can choose between two authentication methods. Both grant the Encore platform the same level of access — the difference is in how that trust is established. **Encore Service Account (recommended)** The Encore platform uses its own Google-managed service account, which you grant access to your GCP Organization. No credentials are ever shared with the Encore platform, and access can be revoked at any time by removing the IAM binding. This requires that the Encore platform service account is allowed by your Organization's `Domain restricted sharing` policy. If your organization enforces this policy, you'll need to edit it to allowlist the Encore platform service account (see the Troubleshooting section below). Choose this option if: - You can modify the `Domain restricted sharing` policy in your GCP Organization, or it isn't enforced. - You prefer not to manage long-lived credentials. **Service Account JSON Key** You create a service account in your own GCP project, grant it the required roles, and upload its JSON key to the Encore platform. The Encore platform then authenticates as that service account when managing your infrastructure. This avoids any changes to your `Domain restricted sharing` policy, but means a long-lived credential is stored by the Encore platform. The key should be rotated periodically, and revoking access requires deleting the key in GCP. Choose this option if: - You're unable to modify the `Domain restricted sharing` policy. - You prefer to keep the service account fully within your own GCP Organization. ### Troubleshooting **I can't access/edit the `Policy for Domain restricted sharing` page** To edit Organization policies, you need to have the `Organization Policy Administrator` role. If you don't have this role, you can ask your GCP Organization Administrator to grant you the necessary permissions. If you're a GCP Organization Administrator, you can grant yourself the necessary permissions by following the steps below: 1. Go to the [IAM & Admin page](https://console.cloud.google.com/iam-admin/iam) in the GCP Console. 2. Find your user account in the list of members. 3. Click the pencil icon to edit your user account. 4. Add the `Organization Policy Administrator` role to your user account. 5. Click Save. **I can't grant access to the Encore platform service account** If you're unable to grant access to the Encore platform service account, you may have failed to add the Encore platform to your `Domain restricted sharing` policy. Make sure you've followed all the steps in the Connect Cloud page to add the Encore platform to the policy. If you're using several GCP accounts, make sure you're logged in with the correct account and that the correct organization is selected in the GCP Console. **Encore platform returns "Could not find Organization ID"** If you see this error message, it means that the Encore platform was unable to connect to your GCP Organization. Make sure you've followed all the steps in the Connect Cloud page to grant the Encore platform access to your GCP Organization. If you're using several GCP accounts, make sure you're logged in with the correct account and that the correct organization is selected in the GCP Console. Still having issues? Drop us an email at [support@encore.dev](mailto:support@encore.dev) or chat with us in the [Encore Discord](https://encore.dev/discord. ## Amazon Web Services (AWS) ### Permissions scoping For a seamless experience, the default setup uses an IAM Role that gives the Encore platform the permissions needed to provision and manage infrastructure in your AWS account. The simplest way to scope this is to use a dedicated AWS sub-organization for the Encore platform, which provides clear isolation. It's also possible to configure a more narrowly scoped IAM policy. The required permissions depend dynamically on the structure of your applications and the infrastructure resources they use. We're actively working on providing more solutions for scoping down permissions further. [Contact us](https://encore.dev/book) to discuss the best setup for your needs. ### Setup To configure your Encore app to deploy to your AWS account, head over to the Connect Cloud page by going to the **[the Encore platform dashboard](https://app.encore.dev/) > (Select your app) > App Settings > Integrations > Connect Cloud**. Follow the instructions to create an IAM Role, and then connect the role with the Encore platform. [Learn more in the AWS docs](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user.html). ![Connect AWS account](https://encore.dev/assets/docs/connectaws.png "Connect AWS account") > **Warning** > > For your security, make sure to check `Require external ID` and specify the > external ID provided in the instructions. After connecting your app to AWS, you will be asked to choose which region you want the Encore platform to provision resources in. [Learn more about AWS regions here](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/). --- Source: https://encore.dev/docs/platform/deploy/environments Section: Deploying # Creating & configuring environments > Get the environments you need, without the work Encore automatically sets up and manages different environments for your application (local, preview, testing, and production). Each environment is: - Fully isolated - Automatically provisioned - Always in sync with your codebase - Configured with appropriate infrastructure for its purpose ## Your first environment Every app starts with an environment named `staging`. It's a `development` environment on Encore Cloud's free hosting, it deploys every push to your app's main branch, and its API is served at `https://staging-.encr.app`. ## Environment Types Encore has four types of environments: - `production` - `development` - `preview` - `local` Some environment types differ in how infrastructure is provisioned: - `local` is provisioned by Encore's Open Source CLI using local versions of infrastructure. - `preview` environments are provisioned in the Encore platform hosting and are optimized to be cost-efficient and fast to provision. - `production` and `development` environments are provisioned by the Encore platform, either in your [cloud account](https://encore.dev/docs/platform/deploy/own-cloud) or using the Encore platform's free development hosting. Both environment types offer the same infrastructure options when deployed using your own cloud account. Environment type is also used for [Secrets management](https://encore.dev/docs/ts/primitives/secrets), allowing you to configure different secrets for different environment types. Therefore, you can easily configure different secrets for your `production` and `development` environments. ## Creating environments 1. Open your app in the [Encore platform dashboard](https://app.encore.dev) 2. Go to **Environments** > **Create env** 3. Configure your environment: - Name your environment - Choose type: **Production** or **Development** (see [Environment Types](#environment-types)) - Set deploy trigger: Git branch or manual - Configure infrastructure approval: automatic or manual - Select cloud provider - Choose process allocation: single or separate processes ![Creating an environment](https://encore.dev/assets/docs/createenv.png "Creating an environment") ### Configuring deploy trigger When using GitHub, you can configure the Encore platform to automatically trigger deploys when you push to a specific branch name. To configure which branch name is used to trigger deploys, open your app in the [Encore platform dashboard](https://app.encore.dev) and go to the **Overview** page for your intended environment. Click on **Settings** and then in the section **Branch Push** configure the `Branch name` and hit save. ### Configuring infrastructure approval For some environments you may want to enforce infrastructure approval before deploying. You can configure this in the **Settings** > **Infrastructure Approval** section for your environment. When infrastructure approval is enabled, an application **Admin** will need to manually approve the infrastructure changes before the deployment can proceed. ### Configuring process allocation The Encore platform offers flexible process allocation options: - **Single process**: All services run in one process (simpler, lower cost) - **Separate processes**: Each service runs independently (better isolation, independent scaling) Choose your preferred deployment model when creating each environment. You can use different models for production and development environments without changing any code. ![Microservices - Process Allocation](https://encore.dev/assets/docs/microservices-process-allocation.png) ## Setting a Primary environment Every Encore app has a configurable Primary environment that serves as the default for: - App insights in the Encore platform dashboard - API documentation - CLI functionality (like API client generation) **Configuring your Primary environment:** 1. Open your app in the [Encore platform dashboard](https://app.encore.dev) 2. Navigate to **Settings** > **General** > **Primary Environment** 3. Select your desired environment from the dropdown 4. Click **Update** ## Redirecting HTTP to HTTPS Requests to your environment, both on its `encr.app` domain and on any [custom domains](https://encore.dev/docs/platform/integrations/custom-domains), go through Encore's edge before they reach your app. The edge can redirect requests made over plain HTTP to the same URL on HTTPS, so every service is covered without checks in your own code. It responds with `301 Moved Permanently` to `GET` and `HEAD` requests, and with `308 Permanent Redirect` to other methods, so clients repeat the request with the same method and body. The redirect is on by default in environments created after the setting was introduced. In older environments it's off until you turn it on. To change it: 1. Open the **Infrastructure** page for your environment in the [Encore platform dashboard](https://app.encore.dev). 2. Under **Routes**, find your environment's domain and click **Edit** next to it. 3. Turn **Redirect HTTP to HTTPS** on or off and click **Save**. 4. Under **Pending infrastructure changes**, click **Confirm & deploy** to apply it. Custom domains follow the setting of the environment's `encr.app` domain. > **Note** > > Your app can't redirect HTTP requests itself. With the redirect off, the edge sends plain HTTP requests on to your app over HTTPS, so headers like `X-Forwarded-Proto` and `Forwarded` say `https` even when the client used plain HTTP. --- Source: https://encore.dev/docs/platform/deploy/preview-environments Section: Deploying # Preview Environments > Accelerate development with isolated test environments for each Pull Request When using [the Encore platform Pro](https://encore.dev/pricing), you automatically get ephemeral Preview Environments for each Pull Request. Preview Environments are free, fully-managed development environments that are hosted on Encore Cloud. They let you test changes without managing infrastructure or incurring cost. ### Self-hosted Preview Environments (Enterprise) On the Enterprise tier, Preview Environments can also be hosted on your own cloud infrastructure. This is useful when your Preview Environments need network access to private resources, such as shared databases or internal APIs. For example, on GCP you can isolate all Preview Environments to a single GCP project, so you only need to set up networking and VPC access to your other private resources once and all Preview Environments will automatically have access. [Contact us](https://encore.dev/book) to discuss your requirements. See the [infra docs](https://encore.dev/docs/platform/infrastructure/infra#preview-environments) if you're curious about exactly how Preview Environments are provisioned. ## Using Preview Environments To use Preview Environments, you first need to [connect your application to GitHub](https://encore.dev/docs/platform/integrations/github). Preview Environments are named after the pull request, for example PR #72 creates a Preview Environment named `pr:72` with the API base url `https://pr72-$APP_ID.encr.app`. You can also view the environment in the Encore platform dashboard, where the url will be `https://app.encore.dev/$APP_ID/envs/pr:72`. ![Preview environment linked in GitHub](https://encore.dev/assets/docs/ghpreviewenv.png "Preview environment linked in GitHub") ## Populate databases with test data automatically Preview Environments can automatically come with pre-populated test data thanks to Neon's database branching feature. Here's how it works: 1. Your main database (typically in staging) contains your test data 2. When a Preview Environment is created, it gets a fresh database that's an exact copy of your main database 3. This happens automatically - no manual data copying needed! #### Setup instructions 1. Go to [the Encore platform dashboard](https://app.encore.dev) 2. Select your app > App Settings > Preview Environments 3. Choose which environment's database to copy from (e.g., staging) 4. Save your changes **Note:** This feature requires using Neon as your database provider, which is: - Default for Encore Cloud environments - Optional for AWS and GCP environments ![Use Neon for PR environments](https://encore.dev/assets/docs/pr-neon.png) ## Frontend Collaboration Preview Environments make it really easy to collaborate and test changes with your frontend. Just update your frontend API client to point to the `pr:#` environment. This is a one-line change since your API client always specifies the environment name, e.g. `https://-.encr.app/`. If your pull request makes changes to the API, you can [generate a new API client](https://encore.dev/docs/ts/cli/client-generation) for the new backend API using `encore gen client --env=pr:72 --lang=typescript my-app` --- Source: https://encore.dev/docs/platform/deploy/private-packages Section: Deploying # Private Packages > Install private npm packages and Go modules in your builds If your app depends on private packages, such as an npm scope on GitHub Packages or Go modules in a private GitHub organization, the Encore Cloud build needs a token to fetch them. You add that token in the Cloud Dashboard as a **private package credential**, and every build of the app uses it from then on. The token is encrypted and never enters the build: the build host fetches private packages on your build's behalf and authenticates to the registry itself. Your code, build hooks, tests and config files never see the token, so you don't need an `NPM_TOKEN` secret or a token in your repository. ## Adding a credential Open your app in the [Cloud Dashboard](https://app.encore.dev) and go to **Settings > Builds**. In the **Private packages** section, click **Add credential** and choose what it is for: * **npm package scope** for private npm packages * **Go modules** for private Go modules Credentials apply right away, to every build of the app, including production. There is one credential per npm scope or Go module pattern. ### npm package scopes | Field | Description | | --- | --- | | **Scope** | The npm scope, like `@acme`. Every package in the scope is installed from the registry below. | | **Registry URL** | The `https://` URL the scope is served from, like `https://npm.pkg.github.com` for GitHub Packages. | | **Username** | Optional. Set it if the registry needs a username and password rather than a token. | | **Token** | The registry token. For GitHub Packages, use a classic personal access token with the `read:packages` scope. | The credential takes precedence over your project's own configuration for that scope. If your repository already has an `.npmrc` (or `.yarnrc.yml`) that points the scope at the registry with a `${NPM_TOKEN}` placeholder, you can leave it as it is: the build routes the scope through the build host regardless, and committed lockfiles that record the registry's tarball URLs keep working with `npm ci` and frozen lockfiles. Other settings in your configuration are left alone. This works with npm, pnpm, Yarn (Classic and Berry) and Bun. > **Note** > > With pnpm 11.0–11.9, `registries` set in `pnpm-workspace.yaml` take precedence over the credential. The same goes for Bun's `[install.scopes]` in `bunfig.toml`. Configure the scope in `.npmrc` instead, or leave it out and let the credential provide it. ### Go modules | Field | Description | | --- | --- | | **Module path pattern** | A [`GOPRIVATE`](https://go.dev/ref/mod#private-modules)-style pattern that starts with the host name, like `github.com/acme/*` or `github.com/acme/internal-lib`. | | **Username** | Optional. Set it if the host needs a specific login with the token. | | **Token** | A token with read access to the repositories. For GitHub, a fine-grained or classic personal access token that can read the repositories' contents. | Modules matching the pattern are fetched directly from their repository host with the token, and are not checked against the public checksum database. Everything else is fetched through the public Go module proxy as usual. You don't need to set `GOPRIVATE` or configure git credentials in your build. You can add several credentials for the same host, for example one token for `github.com/acme/*` and another for `github.com/partner/*`, which is what GitHub's fine-grained tokens require when your modules span organizations. The most specific pattern wins. Credentials are told apart by the part of the pattern before the first wildcard, so two patterns that share it, like `github.com/acme/foo-*` and `github.com/acme/bar-*` (both `github.com/acme`), can't both be added. To keep separate tokens for different repositories in one organization, name the repositories instead, like `github.com/acme/foo-api` and `github.com/acme/bar-lib`. ## Updating and removing credentials To rotate a token, click **Edit** next to the credential and enter the new token. Tokens are write-only: the dashboard shows what each credential is for and when it was last changed, but never the token itself. Leave the token field empty to keep the current one when changing other fields. The scope or pattern of a credential can't be changed; remove it and add a new one instead. Click **Remove** to delete a credential; it takes effect from the next build. Builds that need the private packages fail until you add a new one. ## Permissions Because a credential reaches every build of the app, including production, only app owners can add, change or remove credentials. Everyone with access to the app can see which credentials exist, but never their tokens. With [custom roles](https://encore.dev/docs/platform/management/permissions#custom-roles--permissions) you can grant these permissions separately: *List private package credentials* and *Add, change and remove private package credentials*. --- Source: https://encore.dev/docs/platform/infrastructure/infra Section: Deploying # Infrastructure provisioning & Environments > How the Encore platform provisions infrastructure for your application The Encore platform automatically provisions all necessary infrastructure, in all environments and across all major cloud providers, without requiring application code changes. You simply [connect your cloud account](https://encore.dev/docs/platform/deploy/own-cloud) and create an environment. ## How it works This is powered by Encore's open source [backend framework](https://encore.dev/docs/ts), which lets you declare infrastructure resources (databases, caches, queues, scheduled jobs, etc.) as type-safe objects in application code. At compile time, Encore parses the application code to generate an [Application Model](https://encore.dev/docs/understanding-encore), and the Encore platform uses this meta data to create an infrastructure graph with a high-resolution definition of the infrastructure your application requires. The Encore platform then uses this graph to provision and manage the necessary infrastructure in your cloud account (using AWS and GCP APIs), and in development and preview environments hosted on Encore Cloud. ![Infrastructure Provisioning](https://encore.dev/assets/docs/howitworks.png) The approach removes the need for infrastructure configuration files and avoids creating cloud-specific dependencies in your application. Having an end-to-end integration between application code and infrastructure also enables the Encore platform to keep environments in sync and track cloud infrastructure, giving you an up-to-date view of your infrastructure to avoid unnecessary cloud costs. ![Infrastructure Tracking](https://encore.dev/assets/docs/infra_config_new.png) ## Environment types By default, the Encore platform provisions infrastructure using contextually appropriate objectives for each environment type. You retain control over the infrastructure in your cloud account, and can configure it directly both via the Encore platform dashboard and your cloud provider's console. The Encore platform takes care of syncing your changes. | | Local | Encore platform Hosting | GCP / AWS | | ---------------------- | ------------------ | -------------------------- | ---------------------------------- | | **Environment types:** | Development | Preview, Development | Development, Production | | **Objectives:** | Provisioning speed | Provisioning speed, Cost\* | Reliability, Security, Scalability | \*the Encore platform Hosting is free to use, subject to Fair Use guidelines and usage limits. [Learn more](https://encore.dev/docs/platform/management/usage) ## Development Infrastructure The Encore platform provisions infrastructure resources differently for each type of development environment. | | Local | Preview / Development (Encore platform Hosting) | GCP / AWS | | ------------------- | --------------------------------- | ------------------------------------------------------------ | -------------------------------------------------------------- | | **SQL Databases:** | Docker | Encore platform Managed (Kubernetes), [Neon](https://encore.dev/docs/platform/infrastructure/neon) | [See production](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) | | **Pub/Sub:** | In-memory ([NSQ](https://nsq.io)) | GCP Pub/Sub | [See production](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) | | **Caches:** | In-memory (Redis) | In-memory (Redis) | [See production](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) | | **Cron Jobs:** | Disabled | [Encore platform Managed](https://encore.dev/docs/primitives/cron-jobs) | [See production](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) | | **Object Storage:** | Local Disk | Encore platform Managed | [See production](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) | ### Local Development For local development the Encore platform provisions a combination of Docker and in-memory infrastructure components. SQL Databases are provisioned using [Docker](https://docker.com). For Pub/Sub and Caching the infrastructure is run in-memory. When running tests, a separate SQL Database cluster is provisioned that is optimized for high performance (using an in-memory filesystem and fsync disabled) at the expense of reduced reliability. To avoid surprises during development, Cron Jobs are not triggered in local environments. They can always be triggered manually by calling the API directly from the [development dashboard](https://encore.dev/docs/dev-dash). The application code itself is compiled and run natively on your machine (without Docker). ### Preview Environments When you've [connected your application to GitHub](https://encore.dev/docs/platform/integrations/github), the Encore platform automatically provisions a temporary [Preview Environment](https://encore.dev/docs/platform/deploy/preview-environments) for each Pull Request. Preview Environments are created in the Encore platform Hosting by default, and are optimized for provisioning speed and cost-effectiveness. The Preview Environment is automatically destroyed when the Pull Request is merged or closed. On the Enterprise tier, Preview Environments can also be hosted on your own cloud infrastructure. See the [Preview Environments docs](https://encore.dev/docs/platform/deploy/preview-environments#self-hosted-preview-environments-enterprise) for more information. Preview Environments are named after the pull request, so PR #72 will create an environment named `pr:72`. ### Encore Cloud Hosting Encore Cloud Hosting is a simple, zero-configuration hosting solution provided by Encore. It's perfect for development environments and small-scale use that do not require any specific SLAs. It's also a great way to evaluate the Encore platform without needing to connect your cloud account. Encore Cloud Hosting is not designed for business-critical use and does not offer reliability guarantees for persistent storage like SQL Databases. Other infrastructure primitives like Pub/Sub and Caching are provisioned with small-scale use in mind. [Learn more about the usage limitations](https://encore.dev/docs/platform/management/usage) ## Production Infrastructure The Encore platform provisions production infrastructure resources using best-practice guidelines and services for each respective cloud provider. | | GCP | AWS | | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | **Networking:** | [VPC](https://encore.dev/docs/platform/infrastructure/gcp#networking-architecture) | [VPC](https://encore.dev/docs/platform/infrastructure/aws#networking-architecture) | | **Compute:** | [Cloud Run](https://encore.dev/docs/platform/infrastructure/gcp#google-cloud-run), [GKE](https://encore.dev/docs/platform/infrastructure/gcp#google-kubernetes-engine) | [Fargate ECS](https://encore.dev/docs/platform/infrastructure/aws#aws-fargate), [EKS](https://encore.dev/docs/platform/infrastructure/aws#amazon-eks) | | **SQL Databases:** | [GCP Cloud SQL](https://encore.dev/docs/platform/infrastructure/gcp#databases), [Neon](https://encore.dev/docs/platform/infrastructure/neon) | [Amazon RDS, Amazon Aurora](https://encore.dev/docs/platform/infrastructure/aws#databases), [Neon](https://encore.dev/docs/platform/infrastructure/neon) | | **Pub/Sub:** | [GCP Pub/Sub](https://encore.dev/docs/platform/infrastructure/gcp#pubsub) | [Amazon SQS & Amazon SNS](https://encore.dev/docs/platform/infrastructure/aws#pubsub) | | **Object Storage:** | [GCS/Cloud CDN](https://encore.dev/docs/platform/infrastructure/gcp#object-storage) | [Amazon S3/CloudFront](https://encore.dev/docs/platform/infrastructure/aws#object-storage) | | **Caches:** | [GCP Memorystore (Redis)](https://encore.dev/docs/platform/infrastructure/gcp#caching) | [Amazon ElastiCache (Redis)](https://encore.dev/docs/platform/infrastructure/aws#caching) | | **Cron Jobs:** | Encore platform Managed | Encore platform Managed | Encore platform Managed | | **Secrets:** | [Secret Manager](https://encore.dev/docs/platform/infrastructure/gcp#secrets-management) | [AWS Secrets Manager](https://encore.dev/docs/platform/infrastructure/aws#secrets-management) | ### Configuration With Encore you do not define any cloud service specifics in the application code. This means that after deploying, you can safely use your cloud provider's console to modify the provisioned resources, or use the built-in configuration UI in the Encore platform dashboard. Learn more in the [Infrastructure Configuration](https://encore.dev/docs/platform/infrastructure/configuration) documentation. For information on infrastructure safety, upgrades, disaster recovery, and shared responsibilities, see [Managing Infrastructure](https://encore.dev/docs/platform/infrastructure/managing-infrastructure). --- Source: https://encore.dev/docs/platform/infrastructure/configuration Section: Deploying # Infrastructure Configuration > How to configure infrastructure when using the Encore platform The Encore platform provides a powerful and flexible approach to infrastructure management, ensuring that your cloud resources are efficiently provisioned, according to enterprise best practices. Unlike traditional Infrastructure-as-Code (IaC) tools, when using Encore's declarative infrastructure framework, you do not define any cloud service specifics in code. This ensures your code is cloud-agnostic and portable across clouds, and can be deployed using different infrastructure for each environment according to your priorities (cost, performance, etc.). Infrastructure configuration is made in the Encore platform dashboard, which provides a controlled workflow, role-based access controls, and auditable history of changes. The Encore platform provisions and manages infrastructure by using your cloud provider's APIs. Learn more in the [Infrastructure](https://encore.dev/docs/platform/infrastructure/infra) documentation. ## Infrastructure settings when creating a new environment When creating a new environment, you can decide the following: - Which cloud provider to use (AWS or GCP) - Which compute hardware to use (e.g. AWS Fargate, GCP Cloud Run, Kubernetes) - If using Kubernetes, should a new cluster be created or should an existing cluster be used? - Which Kubernetes provider to use (GKE or EKS) - Which database to use (e.g. AWS RDS, AWS Aurora, GCP CloudSQL, Neon Serverless Postgres) - Which process allocation strategy to use (more on this below) ## Ongoing infrastructure configuration ### Configuration UI in the Encore platform After creating an environment, you can continue to configure the infrastructure via the Encore platform dashboard. The dashboard exposes the most common configuration options, and provides a controlled workflow for making changes, including audit logs and role-based access controls. ![Infra configuration UI](https://encore.dev/assets/docs/infra_config.png) #### Process allocation configuration Encore provides a powerful configuration option called process allocation. This enables you to configure how microservices should be deployed on the compute hardware; either deploying all services in one process or one process per service. All without any code changes. It's often recommended to deploy all services in one process in order to reduce costs and minimize response times between services. (But it depends on your use case.) Deploying each service as its own process will improve scalability and decrease blast radius if things go wrong. This is only recommended for production environments. ![Process allocation config](https://encore.dev/assets/docs/microservices-process-allocation.png) ### Manual configuration in your cloud provider's console Manual configuration is relevant in cases where some configuration options are not yet available in the Encore platform dashboard, or you may want to make changes manually. Handily, you have full access to make changes directly in your cloud provider's console. The Encore platform tries very hard to ensure that any manual changes made in the cloud provider's console are not overwritten. Therefore it only makes the minimum necessary modifications to infrastructure when deploying new changes, using the following strategies: - **PATCH-style updates:** Resources are updated using compare-and-set and similar techniques, modifying only the attributes that require changes. - **Avoid full syncs:** Unlike Terraform, the Encore platform updates only the specific resources necessary to accomplish an infrastructure change rather than performing a complete infrastructure refresh. - **Drift-aware updates:** Before making any changes to a resource, the Encore platform pulls the current resource properties. If drift is detected (e.g. a setting was changed directly in the cloud console), Encore updates its internal representation to match the current state, unless there is a pending, unapplied change request in the Encore platform dashboard. These behaviors ensure an efficient and predictable workflow, minimizing unintended changes and reducing deployment times, and means that you can safely use your cloud provider's console to modify the provisioned resources. This behavior also makes the Encore platform well-suited for environments where infrastructure is partially managed outside of the Encore platform, enabling you to deploy Encore applications alongside existing infrastructure (more on this below). ## Working with Existing Infrastructure One of the Encore platform’s strengths is its ability to work seamlessly with existing infrastructure. Since it does not enforce a full sync approach, it can: - Integrate with pre-existing cloud resources without overwriting manual changes - Deploy to existing Kubernetes clusters - Co-exist with other IaC tools like Terraform and CloudFormation. The Encore platform also provides a Terraform Provider to simplify integration with existing Terraform-managed infrastructure. Learn more in the [Terraform Provider](https://encore.dev/docs/platform/integrations/terraform) documentation. --- Source: https://encore.dev/docs/platform/infrastructure/tagging Section: Deploying # Resource Tagging > How Encore identifies and groups your cloud resources Encore adds identifying metadata to supported infrastructure resources as it provisions them: **tags on AWS** and **labels on GCP**. These connect a cloud resource to the application, environment, and resource it represents in Encore's infrastructure graph. Some GCP resources also carry the names of the Encore services they host or support. This metadata is useful for several common tasks: - **Finding resources:** filter your cloud inventory to see the infrastructure associated with an application or environment. - **Investigating issues:** identify which Encore environment a database, compute service, or network resource belongs to when inspecting it in the cloud console. - **Organizing infrastructure:** group related resources and connect their application and environment IDs to your own ownership, team, or cost-center records. - **Analyzing costs:** group supported billing data by application, environment, or individual resource, and identify infrastructure shared by multiple services. ## Common identifiers AWS and GCP use the same five core keys and value formats on the supported resources described below. Example IDs are abbreviated; actual values contain the full ID. | Key | Meaning | Example | | --------------- | ---------------------------------------------------------- | ------------------ | | `encore-app` | Your application's slug, lowercased | `my-app` | | `encore-app-id` | Encore's unique application ID | `app_…` | | `encore-env` | Your environment's name, converted to lowercase kebab-case | `preview-checkout` | | `encore-env-id` | Encore's unique environment ID | `env_…` | | `encore-id` | The resource's ID in Encore's infrastructure graph | `res_…` | Use names for readable filters and IDs for durable references. Application and environment IDs remain useful across name changes; recreating an environment produces a different identity even if you reuse its name. `encore-env` is the environment's name, not a classification such as production or development. For those classifications or business ownership, you can maintain a lookup keyed by application and environment IDs. These IDs identify Encore entities and are separate from AWS ARNs, GCP project IDs, and cloud resource names. An infrastructure resource can host several logical resources: for example, one database server can contain several application databases. Its `encore-id` identifies that server in Encore's graph. Tagging is implemented per resource type. Membership in the infrastructure graph does not automatically mean a cloud resource receives tags, and tags on a parent should not be assumed to appear on every child. The sections below describe the current coverage and provider-specific behavior. ## AWS Encore applies the five core identifiers as AWS resource tags. The AWS implementation currently does **not** add the `encore-services` tag described in the GCP section. ### Resource coverage | AWS resources | Encore tags applied | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | | ECS clusters, services, and task definitions used for Fargate | Core tags and `Name` | | EKS clusters and Fargate profiles | Core tags | | RDS DB instances, Aurora DB clusters and their DB instances, DB subnet groups, and parameter groups | Core tags | | ElastiCache Redis replication groups, subnet groups, parameter groups, users, and user groups | Core tags | | SNS topics and SQS queues | Core tags | | ECR repositories | Core tags | | Application Load Balancers, target groups, and listeners | Core tags | | CloudFront distributions | Core tags | | ACM certificates | Core tags and `Name` | | Secrets Manager secrets and KMS keys | Core tags | | IAM roles, customer-managed policies created by Encore, and OIDC providers | Core tags | | VPCs, subnets, internet gateways, NAT gateways, Elastic IPs, VPC peering connections, route tables, network ACLs, security groups, and ingress/egress security-group rules | Core tags and `Name`; selected EKS resources also receive integration tags | | AWS Resource Groups created by Encore | Core tags and `Name` | This lists directly tagged resource types. It does not imply that every object within those services receives tags. For example, the current S3 bucket and object implementations do not apply Encore's core tags. Logical databases inside RDS are represented by the database instance's tags. ### Names and environment resource groups On selected resources, Encore adds the AWS `Name` tag to make them easier to recognize in cloud tools. It normally contains the generated resource name; for ACM certificates it contains the domain name, and for security-group rules it contains the rule description. Encore also creates an AWS Resource Group for each environment's region. The group's query selects resources whose `encore-env-id` matches that environment, across the resource types supported by AWS Resource Groups. This gives you an environment-oriented inventory in the AWS console. Its membership depends on both the tags present and [AWS Resource Groups' supported resource types](https://docs.aws.amazon.com/ARG/latest/userguide/supported-resources.html). ### ECS task propagation Encore configures ECS services to propagate **task-definition tags** to their tasks and enables ECS-managed tags. Consequently, a task's propagated `encore-id` identifies the Encore task-definition resource; it is not a unique Encore ID for each running task. Use the ECS task ARN to distinguish individual executions. ### EKS integration tags and workload labels Encore adds tags needed by the AWS Load Balancer Controller on selected EKS networking resources: | Tag | Value and purpose | | -------------------------------------- | --------------------------------------------------------------------------- | | `kubernetes.io/role/elb` | `1` on public subnets used for external load balancers | | `kubernetes.io/role/internal-elb` | `1` on cluster subnets used for internal load balancers | | `kubernetes.io/cluster/` | `shared` on participating subnets and `owned` on the cluster security group | These tags support the Kubernetes integration in addition to the core Encore identifiers. Inside EKS, Encore uses Kubernetes labels on Deployments and their Pods. The core keys become `encore.app/app`, `encore.app/app-id`, `encore.app/env`, `encore.app/env-id`, and `encore.app/id`. An additional `encore.app/deployment` label contains the Kubernetes Deployment name. Here, `encore.app/id` identifies the Encore Deployment resource. These are Kubernetes object labels, separate from AWS resource tags; the deployment labeling path does not generate an `encore-services` equivalent. For workloads using an EKS Fargate profile, Encore also adds `encore.app/eks-fargate-profile` with the profile's name. ### Using AWS tags for costs To use Encore tags in AWS cost reporting, activate the relevant keys as **user-defined cost allocation tags** in AWS Billing and Cost Management. Tagging a resource and activating a key for billing are separate steps. Start with `encore-app-id` and `encore-env-id` for application and environment breakdowns. See [AWS's cost allocation tag documentation](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/custom-tags.html). ## GCP Encore applies the five core identifiers as **GCP labels**, including Cloud SQL's user labels. These are distinct from Resource Manager tag bindings and network tags. GCP labels are not inherited through the resource hierarchy: labeling a project does not automatically label its resources. See [Google's comparison of tags and labels](https://docs.cloud.google.com/resource-manager/docs/tags/tags-overview). ### Resource coverage | GCP resources | Encore labels applied | | --------------------------------------------------------------------------------------- | --------------------------------------------------------- | | Cloud Run services and their revision templates | Core labels; `encore-services` for hosted Encore services | | Cloud SQL instances | Core labels and associated services, when available | | Memorystore Redis instances | Core labels and associated services, when available | | Cloud Storage buckets | Core labels and associated services, when available | | Pub/Sub topics and subscriptions | Core labels and associated services, when available | | Secret Manager secrets | Core labels | | Projects created by Encore | Core labels | | GKE clusters | Core labels | | Global load-balancer forwarding rules | Core labels | | Certificate Manager certificates, certificate maps, map entries, and DNS authorizations | Core labels | The current implementation does not directly apply these GCP labels to: - VPC networks, subnets, VPC peerings, and peering ranges. - Reserved external and internal IP addresses and Serverless VPC Access connectors. - Private Service Connect endpoints and forwarding rules. - Load-balancer URL maps, target HTTP(S) proxies, and backend buckets. - Artifact Registry repositories and container images. - IAM service accounts, roles, and bindings. - GKE node pools and Kubernetes namespaces. - Cloud SQL databases and users inside an instance. Logical databases within Cloud SQL and keyspaces within Redis are represented by their parent instance's labels. ### Projects and environment organization With a GCP organization connection, Encore's default is to create a project for the environment. Its generated name includes application and environment information, and the project carries the core labels. You can use that project to find the environment's resources, including those without individual labels. Encore does not automatically label imported GCP projects. For other imported resources, label coverage depends on how Encore manages them. Existing projects and infrastructure configuration can change this arrangement. An imported project may contain resources from other environments or workloads managed outside Encore. Before treating a project's inventory or costs as belonging to one environment, verify that the project is dedicated to it. Project labels describe the project; they do not establish ownership of every resource inside it. ### Adding your own labels You can add metadata such as `team` or `cost-center`, but whether Encore preserves it depends on the resource's update behavior: | Resource | Behavior for additional labels | | ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | GCP projects | Encore preserves customer-added labels. | | Cloud Run services and revision templates | Encore merges its labels into the existing labels. | | Cloud Storage buckets | Encore preserves additional labels. | | Cloud SQL instances | Encore preserves additional user labels. | | Memorystore Redis instances, Pub/Sub topics and subscriptions, and GKE clusters | Encore replaces the label set during label updates, so additional labels can be removed. | For provider details, see Google's [Cloud Storage label documentation](https://docs.cloud.google.com/storage/docs/using-bucket-labels) and [Cloud SQL label documentation](https://docs.cloud.google.com/sql/docs/postgres/label-instance). Encore still supplies its own values for the core keys, so use separate keys for your metadata. To maintain business categories consistently across resource types, keep a lookup keyed by `encore-app-id` and `encore-env-id` in your inventory or analytics system. ### Associated services: `encore-services` On selected resources, Encore adds `encore-services` to describe the services they host or support. The value comes from the application's infrastructure graph: | Resource | Services included | | ----------------------------- | --------------------------------------------------------------------------- | | Cloud Run application service | Encore services hosted together in that Cloud Run service | | Cloud SQL instance | Services linked to any of the Encore databases hosted on the instance | | Memorystore Redis instance | Services linked to any of the Encore cache keyspaces hosted on the instance | | Cloud Storage bucket | Services linked to the bucket, regardless of public access or CDN settings | | Pub/Sub topic | Services linked as publishers of the topic | | Pub/Sub subscription | Services linked to that subscription | Encore joins the names with underscores and lowercases the result. For data resources, it first deduplicates and sorts the linked service names. For example, a Cloud SQL instance used by `orders` and `payments` receives `encore-services=orders_payments`. If the joined value exceeds 63 bytes, Encore uses `many-services`. If no associated services are available, it does not generate this label. Examples include dead-letter topics created for subscriptions and Cloud Run services used only for supporting infrastructure. These resources can be in use even without a direct association to an Encore service. The label describes structural relationships, not every service that might indirectly cause work through another service. Topic and subscription labels can differ because publishers and subscribers are represented separately. Associations can also change when you change your application's architecture or which services share compute or storage. Treat the value as a grouping key rather than a lossless inventory: names are lowercased, underscores separate them, and long values collapse to `many-services`. For cost analytics, a shared label identifies a shared resource's cost; it does not measure each service's share. Count the resource once or apply an explicit allocation rule. ### GKE workload labels For Kubernetes Deployments and their Pods, Encore uses the same Kubernetes labels as on EKS: | Kubernetes label | Meaning | | ----------------------- | ---------------------------------------------------- | | `encore.app/app` | Application slug | | `encore.app/app-id` | Application ID | | `encore.app/env` | Environment name | | `encore.app/env-id` | Environment ID | | `encore.app/id` | Encore infrastructure resource ID for the Deployment | | `encore.app/deployment` | Kubernetes Deployment name | These labels identify workloads within the cluster. The deployment labeling path does not generate an `encore-services` equivalent; use the deployment name or resource ID for workload grouping. Encore defaults GKE cost allocation to enabled for newly configured clusters, although cluster configuration can override it. With GKE cost allocation and the detailed Cloud Billing export enabled, Google exposes Pod labels under `k8s-label/` in the export's `labels` field. For example, `encore.app/id` becomes `k8s-label/encore.app/id`. For workload attribution, prefer that key over `encore-id` if both exist, since the latter may identify the cluster. Allocation covers supported SKUs and can leave overhead and unallocated capacity outside your workloads. See [Google's GKE cost allocation guide](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/cost-allocations). ### Using GCP labels for costs In Cloud Billing Reports, group by an available label such as `encore-app-id` or `encore-env-id`. In the billing export, resource labels appear in `labels`, while project labels appear separately in `project.labels`. See the [Cloud Billing export schema](https://docs.cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage). A label on a resource does not guarantee that every related charge carries it. Keep unattributed costs visible alongside labeled costs. Label changes affect future exported usage data; previously exported records retain their historical values. See [Cloud Billing export behavior](https://docs.cloud.google.com/billing/docs/how-to/export-data-bigquery-tables). --- Source: https://encore.dev/docs/platform/infrastructure/neon Section: Deploying # Use Neon Postgres [Neon](https://neon.tech/) is a serverless database provider that offers a fully managed and autoscalable Postgres database. You can configure the Encore platform to provision a Neon Postgres database instead of the default offering for all supported cloud providers. ## Connect your Neon account To start using Neon with the Encore platform, you need to add your Neon API key to your Encore application. You can sign up for a Neon account at [neon.tech](https://neon.tech/). Once you have an account, you can find your API key in the [Neon Console](https://neon.tech/docs/manage/api-keys) Then, head over to the Neon settings page by going to the [the Encore platform dashboard](https://app.encore.dev) > (Select your app) > App Settings > Integrations > Neon. Click the "Connect Account" button, give it a name, and enter your API key. ![Connect Neon Account](https://encore.dev/assets/docs/connect-neon.png) ## Creating environments using Neon Neon organizes databases in projects. A project consist of a main branch and any number of feature branches. [Branches](https://neon.tech/docs/introduction/branching) in Neon are similar to branches in git, letting you to create a new branch for each feature or bug fix, to test your changes in isolation. When configuring your Encore environment to use Neon, you can choose which project and branch to use. To get started, head to the [Encore platform dashboard](https://app.encore.dev) > (Select your app) > Environments > Create Environment. In the Database section, select `Neon database`. ![Create Neon Environment](https://encore.dev/assets/docs/create-neon.png) ### Create a new Neon project and branch If you're starting off a blank slate, you can let the Encore platform create a new Neon project and branch for you. Select `New Neon project` and choose a Neon account and region. We recommend picking a region close to your compute and that you use the suggested project and branch names, but you're free to choose any configuration you like. ### Branch from an existing Encore Cloud environment If you already have an Encore Cloud environment with Neon, you can branch your database from that environment. Simply select `Branch from Encore environment` and choose the environment you want to branch from. This option will be disabled if you don't have any environments using Neon. ### Branch from an existing Neon branch You can also choose to manually select a Neon branch to branch from. This is useful if you have an existing Neon project, but it's not currently being used by any Encore Cloud environments. Select `Branch from Neon project`, then choose the account, project and branch you want to use. ### Import an existing Neon branch The final option is to import an existing Neon branch. This is useful if you have an existing database you want to use. Be wary that this option will not create a new branch but operate on the existing data. Select `Import Neon branch`, then choose the account, project and branch you want to use. **Note:** You may need to manually adjust the roles, commonly you need to change the database owner to the `db__admin` role to enable execution of migrations. See more in the [Roles](#roles) section below. ## Edit your Neon environment Once the environment is created, you can edit the Neon settings by going to the [Encore platform dashboard](https://app.encore.dev) > (Select your app) > Environments > (Select your environment) > Infrastructure. Here you can view and edit your Neon account resources. As a safety precaution, we've disabled editing of imported resources to prevent accidental changes to shared data. ![Edit Neon Environment](https://encore.dev/assets/docs/edit-neon.png) ### Neon project The retention history specifies how long Neon will keep changes to your data. The default is 1 day, but depending on your Neon plan, you can increase this to up to 30 days. ### Neon endpoint Each branch is assigned a unique endpoint which essentially is the serverless compute handling your database. You can edit the endpoint to set the CPU limits and the suspend timeout. The suspend timeout is the time Neon will wait before suspending the compute when it's not in use. The default is 5 minutes, but you can increase this to up to a week (depending on your Neon plan). ## Use Neon for Preview Environments Neon is a great choice for [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments) as it allows you to branch off a populated database and test your changes in isolation. To configure which branch to use for Preview Environments, head to the [the Encore platform dashboard](https://app.encore.dev) > (Select your app) > App Settings > Preview Environments and select the environment with the database you want to branch from. Hit save and you're all done. Keep in mind that you can only branch from environments that use Neon as the database provider; this is the default for Encore Cloud environments, but is a configurable option when creating AWS and GCP environments. ![Use Neon for Preview Environments](https://encore.dev/assets/docs/pr-neon.png) ## Roles The Encore platform automatically implements a structured role hierarchy that ensures a secure, scalable, and efficient management of databases. Below is an explanation of how roles are created, utilized, and managed. ### Role hierarchy #### 1. Initial Superuser Role - **Role Name:** `encore_platform` - **Access level:** This role has full privileges and is the foundational user for setting up the role hierarchy. - **Purpose:** The role creates and configures the subsequent roles and then steps back from day-to-day operations. #### 2. Global Roles Three core roles are created to define access levels across all databases: - `encore_reader` - **Access level:** Provides read-only access. - **Use Case:** Reading data without modifying it. - `encore_writer` - **Access level:** Allows read and write access. - **Use Case:** Performing data manipulations and inserts. - `encore_admin` - **Access level:** Grants administrative privileges for global database operations. - **Use Case:** Overseeing configurations, managing schemas, and handling elevated tasks. These global roles are used by Encore's CLI when using the `encore db shell` command. Learn more in the [CLI docs](https://encore.dev/docs/ts/primitives/databases#using-the-encore-cli). #### 3. Database-Specific Roles For each database within the Neon integration, specific roles are created to provide fine-grained control: - `db__reader`: Read-only access to the main database. - `db__writer`: Read and write access to the main database. - `db__admin`: Administrative privileges specific to the main database. #### 4. Service-Specific Roles For each service in your application, a dedicated role is generated in the format `svc_`. This role is granted the necessary `db__writer` role for each database the service accesses. This ensures that each service has the appropriate level of access to perform its operations while maintaining security and separation of concerns. **Example:** A service named `orders` that writes to the `main` database is assigned the `svc_orders` role, which is granted the `db_main_writer` role. ### Role Setup Workflow - **1. Superuser Creation:** the `encore_platform` superuser role is created upon integration setup. - **2. Global Role Creation:** The `encore_reader`, `encore_writer`, and `encore_admin` roles are established to provide general access control. - **3. Database-Specific Roles:** For each database, roles are created in the format `db__` to manage access specific to that database. - **4. Service-Specific Roles:** For each service, roles are created in the format `svc_` and are granted the necessary writer roles for the databases used by each service. ### Viewing credentials To view database credentials, open your app in the [Encore platform dashboard](https://app.encore.dev), navigate to the **Infrastructure page** for the appropriate **Environment**, and locate the **USERS** section within the relevant **Database Cluster**. ### Best Practices The Encore platform automatically manages roles according to these security best practices: - **Role Ownership:** Ensures critical operations, such as migrations, are executed by roles with appropriate permissions (e.g., `db__admin`). - **Access Control:** Assigns the least privilege necessary for each task. Uses specific database roles (e.g., `db__reader`) to restrict access. - **Consistency:** Maintains consistent naming conventions (`db__`) for ease of management and troubleshooting. ### Integrating with existing Neon databases If you are integrating with an existing Neon database, you may need to manually adjust the roles to work with the Encore platform's role structure. Commonly, the adjustment needed is changing the database owner to the `db__admin` role to enable execution of migrations. --- Source: https://encore.dev/docs/platform/infrastructure/cloudflare Section: Deploying # Cloudflare R2 Buckets The Encore platform simplifies the process of using Cloudflare R2 for object storage by automatically provisioning and managing the necessary infrastructure. This guide provides setup instructions and details on how the Encore platform manages your Cloudflare R2 infrastructure. ## Setup Process ### 1. Cloudflare Account Connection To connect your Cloudflare account to the Encore platform: 1. Create a Cloudflare API token using the **Create Additional Tokens** button in the Cloudflare dashboard 2. Add the following permissions: - Zone > Zone: Read - Zone > DNS: Edit - Account > Workers R2 Storage: Edit 3. Add the token in the Encore platform dashboard: - Navigate to App Settings > Integrations > Cloudflare - Click "Connect Account" - Provide an account name and your API token ### 2. Environment Configuration When creating a new environment: 1. Select your preferred cloud provider 2. Choose "Cloudflare R2" as the object storage provider 3. Configure the following R2-specific settings: - Token: Your Cloudflare API token - Account: Your Cloudflare account - Zone: The domain zone for public bucket URLs - Region: Your preferred R2 storage region ## Managed Features ### Bucket Management The Encore platform provides comprehensive bucket management capabilities that adapt to your application's needs. When you define storage requirements in your application, the Encore platform automatically provisions the necessary R2 buckets with appropriate configurations. Each bucket is created with carefully configured policies and access controls to ensure secure yet efficient access to your stored objects. ### Public Access Configuration When working with public buckets, the Encore platform handles all aspects of public access configuration automatically. Each bucket is assigned a unique subdomain that is automatically provisioned and configured in your DNS settings. The bucket is seamlessly integrated with Cloudflare's global CDN network, ensuring fast content delivery worldwide. The Encore platform also configures optimal caching rules to maximize performance while maintaining appropriate cache invalidation policies. This comprehensive setup ensures your public content is served efficiently and securely through Cloudflare's infrastructure. ### Security Controls The Encore platform implements a comprehensive multi-layered security model to protect your R2 storage. At the bucket level, fine-grained access controls ensure that only authorized services can perform specific operations on each bucket. Each service in your application receives its own unique set of credentials, preventing any unauthorized cross-service access. These credentials are securely distributed to the appropriate services through the Encore platform's built-in secrets management system, which handles the entire credential lifecycle. All these configurations are automatically maintained and updated by the Encore platform as you develop your application, ensuring your infrastructure stays aligned with your application's needs. [cloudflare-r2]: https://developers.cloudflare.com/r2/ [cloudflare-cdn]: https://developers.cloudflare.com/cdn/ --- Source: https://encore.dev/docs/platform/infrastructure/gcp Section: Deploying # GCP Infrastructure > Understanding your application's GCP infrastructure The Encore platform simplifies the process of deploying applications by automatically provisioning and managing the necessary GCP infrastructure. This page provides an overview of the components involved and how they work together to support your applications. _Example of Encore project deployment alongside existing legacy systems on GCP:_ ![Migration options](https://encore.dev/assets/docs/gcp-diagram.png) ## Core Infrastructure Components ### Networking Architecture To ensure maximum security and isolation, the Encore platform provisions a dedicated GCP Project for each environment. This project isolation prevents any potential cross-environment access and enables granular control over resources and permissions. Within each project, all resources are deployed into a private network configuration, where they can only communicate with other resources inside the VPC. This private networking approach significantly reduces the attack surface by preventing direct access from the public internet, with traffic only flowing through designated ingress points. ### Container Management The Encore platform provisions a [Google Container Registry (GCR)][gcp-gcr] to store your application's Docker images. The registry implements comprehensive access controls to ensure only authorized users and services can access and manage container images. Through integration with GCP's Identity and Access Management (IAM), each service is granted the minimum required permissions needed to pull its container images. Additionally, GCR performs automated vulnerability scanning on all container images. As new images are pushed to the registry, they are automatically analyzed for known security vulnerabilities in the operating system and application dependencies. This proactive scanning helps identify potential security issues early in the deployment pipeline, allowing you to maintain a secure application environment. ### Secrets Management The Encore platform's integration with Secret Manager provides comprehensive security and seamless access to sensitive data. All secrets are automatically injected as environment variables into your services, eliminating the need for manual configuration while maintaining security. The secrets are protected using industry-standard encryption both when stored and during transmission between services. To ensure maximum security, Secret Manager implements strict access controls - each service can only access the specific secrets it needs, and all access attempts are logged and audited. ## Compute Options The Encore platform provisions one of two compute platforms for running your application containers, based on your choice: ### Google Cloud Run When using Cloud Run, the Encore platform configures: **Service Deployments** Each service is configured with optimized container settings and health check configurations to ensure reliable operation. Environment variables are automatically injected from Secret Manager to securely provide configuration values. Service discovery integration enables seamless communication between services. **Cloud Run Services** Cloud Run services are configured with zero-downtime deployment strategies, ensuring your application remains available during updates. Each service is integrated with a load balancer to distribute traffic efficiently across instances. Health check grace periods are configured to allow containers adequate time to start up before receiving traffic, preventing premature termination of healthy instances. **IAM Configuration** Each deployment receives its own dedicated service account to ensure proper isolation and security. These service accounts are automatically configured with the minimum required permissions needed for operation. This includes access to pull container images from Google Container Registry, write application logs to Cloud Logging, and interact with assigned GCP resources like Cloud Storage buckets and Pub/Sub topics. The service accounts are also granted permission to read secrets from Secret Manager, enabling secure access to sensitive configuration values. This automated permission management ensures your services have exactly the access they need while following security best practices. ### Google Kubernetes Engine When using GKE, the Encore platform configures: - **Cluster Setup** The Encore platform provisions either GKE Autopilot clusters or standard GKE clusters with managed node pools, both configured to run in private subnets for enhanced security. With Autopilot, GKE automatically manages the underlying infrastructure, while with standard clusters the Encore platform configures and maintains optimized node pools based on your workload requirements. In both cases, the nodes are placed in private subnets to ensure they're not directly accessible from the internet, with all traffic flowing through the load balancer. - **Kubernetes Resources** The Encore platform automatically creates and manages all necessary Kubernetes resources for your application. Each Encore service is deployed as a Kubernetes Deployment, ensuring reliable operation and scaling capabilities. These deployments are backed by service accounts configured with appropriate IAM roles to access GCP resources securely. Sensitive configuration data is stored as Kubernetes Secrets and automatically mounted into the appropriate pods. To enable network connectivity, the Encore platform provisions Kubernetes Service and Ingress resources that integrate with the Google Cloud Load Balancer, providing secure external access to your application endpoints. - **Load Balancer Integration** The Encore platform integrates with Google Cloud Load Balancer to provide secure and reliable access to your applications. The load balancer is configured to distribute traffic across your services while handling SSL/TLS termination. All traffic is automatically encrypted using managed SSL/TLS certificates that are provisioned and renewed automatically. This ensures your application endpoints remain secure and accessible through HTTPS without requiring manual certificate management. - **Monitoring Setup** The Encore platform sets up comprehensive monitoring for your GKE clusters by configuring both metrics collection and log management. Container metrics are automatically collected from each pod and exported to your configured monitoring service, providing detailed insights into resource usage, performance, and application behavior. Additionally, all container logs are seamlessly forwarded to Cloud Logging, enabling centralized log aggregation and analysis. This integrated monitoring approach gives you full visibility into your application's health and performance within the Google Cloud ecosystem. - **Service Accounts** The Encore platform implements a comprehensive service account management system that ensures secure and controlled access to GCP resources. Each service in your application receives its own dedicated service account, providing fine-grained access control and isolation between services. These service accounts are automatically configured with IAM roles that map precisely to the GCP services your application needs to interact with. The permission configuration is handled dynamically based on your application's declared resource usage. For example, if your service needs to access a GCS bucket, the Encore platform automatically grants the minimum required permissions for those specific storage operations. Similarly, when your service needs to publish or subscribe to Pub/Sub topics, connect to databases, or retrieve secrets, the appropriate IAM roles are configured automatically. This automated permission management ensures that each service operates under the principle of least privilege, having access only to the resources it explicitly needs to function. This significantly enhances your application's security posture by minimizing the potential impact of any security breach. All of these configurations are automatically maintained and updated by the Encore platform as you develop your application, ensuring your infrastructure stays aligned with your application's needs. ## Managed Services ### Databases The Encore platform provisions [GCP Cloud SQL][gcp-cloudsql] for PostgreSQL databases, providing a robust and scalable database solution: The Encore platform provisions Cloud SQL instances running the latest PostgreSQL version, ensuring you have access to the newest features and security updates. Each instance starts with the smallest available configuration to optimize costs, while maintaining the ability to automatically scale up resources as your application's needs grow. Data protection is a key priority, with automated daily backups retained for 7 days and point-in-time recovery capabilities. This allows you to restore your database to any moment within the retention period if needed. Security is enforced through strategic placement of databases in private subnets, isolating them from direct internet access. Strict access controls ensure that only authorized services and users can connect to the database instances. ### Pub/Sub The Encore platform implements a robust messaging system using [GCP Pub/Sub][gcp-pubsub]. The system is designed with reliability and security in mind, automatically configuring dead-letter topics to capture and preserve any failed messages for later analysis and debugging. Each service in your application receives precisely scoped IAM permissions for publishing and consuming messages, ensuring secure communication between components while maintaining the principle of least privilege. The Encore platform fully manages all subscriptions and topics, handling the complex setup and ongoing maintenance of your messaging infrastructure, allowing you to focus on your application logic rather than infrastructure management. ### Object Storage The Encore platform leverages [Google Cloud Storage][gcp-gcs] for object storage needs. When you declare storage buckets in your application, the Encore platform automatically provisions them with unique names in GCP. Each service that interacts with storage is configured with precisely scoped permissions, ensuring secure access to only the buckets and operations it requires. For public buckets, the Encore platform integrates with Cloud CDN to optimize content delivery, with each bucket accessible through a unique URL. This comprehensive setup provides secure, efficient, and easily manageable object storage capabilities for your application. ### Caching The Encore platform uses [GCP Memorystore for Redis][gcp-redis] to provide a high-performance caching solution. Each Redis instance starts with the smallest available configuration to optimize costs while maintaining the ability to automatically scale up resources as your application's caching needs grow. The instances are configured in a high-availability setup to ensure your cache remains available and performant even during infrastructure updates or zone outages. Access to the cache is secured through Redis authentication, with credentials automatically managed and rotated by the Encore platform to maintain a strong security posture. ### Cron Jobs The Encore platform provides a streamlined approach to scheduled task execution that prioritizes both simplicity and security. Each cron job is executed through authenticated API requests that are cryptographically signed, ensuring that only legitimate, verified requests can trigger your scheduled tasks. The system includes robust source verification that validates all requests originate from the Encore platform's trusted cron infrastructure. This elegant implementation requires no additional infrastructure components, making it both cost-effective and easy to maintain while providing the reliability and security needed for production workloads. [gcp-vpc]: https://cloud.google.com/vpc [gcp-cloudrun]: https://cloud.google.com/run [gcp-gke]: https://cloud.google.com/kubernetes-engine [gcp-secrets]: https://cloud.google.com/secret-manager [gcp-pubsub]: https://cloud.google.com/pubsub [gcp-gcs]: https://cloud.google.com/storage [gcp-cloudsql]: https://cloud.google.com/sql [gcp-redis]: https://cloud.google.com/memorystore [gcp-gcr]: https://cloud.google.com/container-registry --- Source: https://encore.dev/docs/platform/infrastructure/gcp/import-cloud-sql Section: Deploying # Import an existing Cloud SQL instance > Using your pre-existing database instead of provisioning a new one When deploying applications to your own cloud, the Encore platform can provision all necessary infrastructure—including database instances. However, if you already have a Cloud SQL instance, you can connect your Encore application directly to this existing database. ## Benefits Using an existing Cloud SQL instance allows you to: - Maintain data continuity with your existing systems - Preserve specific database configurations - Utilize familiar database setups without migration ## Importing a Cloud SQL instance Follow these steps to import your existing Cloud SQL instance: 1. Navigate to **Create Environment** in the [Encore platform dashboard](https://app.encore.dev) 2. Select the GCP cloud provider 3. Choose **Import Existing Cloud SQL Instance** 4. Add permissions for the Encore Service Account: - Copy the `Encore GCP Service Account` from the cloud dashboard - Go to your project's IAM page in the GCP Console - Grant the `Owner` role to the `Encore GCP Service Account` 5. Return to the Encore platform dashboard 6. Specify your database's `GCP Project ID` and `Cloud SQL Instance Name` 7. Click the `Resolve` button to validate the instance Once validated, you can create the environment. When you deploy to this environment, the Encore platform will automatically connect your application to your imported Cloud SQL instance rather than provisioning a new database. ## Mapping existing databases to your Encore app To access an existing database in your Encore application, you need to specify the name of the existing database when you declare the database in your app. For example, if you have an existing database called `mydb` you can create a reference to it like so: ```typescript const db = new SQLDatabase("mydb"); ``` ```go sqldb.NewDatabase("mydb", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` ## Applying migrations to existing databases Encore uses a table called `schema_migrations` in the public namespace to keep track of which migrations have been applied. If you import an existing database without that table, Encore will create it for you and apply your migrations in order. If the table already exists, Encore expects it to contain exactly two columns: ``` version bigint dirty boolean ``` If the table exists but has a different schema, you will not be able to import it with Encore at this time. If the table exists with an existing entry, Encore will apply all higher versions in your `migrations` directory to the database. ### Bootstrap migration for existing databases If you have an existing production database that wasn't previously managed by Encore's migration system, you need to set up a **bootstrap migration** so that new environments (dev, staging, etc.) are created with the same schema, while your existing database is not modified. **Step 1:** Dump your existing database schema: ```shell $ pg_dump --schema-only --no-owner --no-privileges your_database > 001_bootstrap.up.sql ``` Review the output and clean it up as needed (remove any database-specific settings, comments, etc.), then place it in your service's `migrations` directory. **Step 2:** In your **existing** imported database, manually create the `schema_migrations` table and mark the bootstrap migration as already applied: ```sql CREATE TABLE IF NOT EXISTS schema_migrations (version bigint NOT NULL, dirty boolean NOT NULL); INSERT INTO schema_migrations (version, dirty) VALUES (1, false); ``` This tells Encore that migration `1` (the bootstrap) has already been applied to this database. Encore will skip it and only run migrations with a higher version number. **Step 3:** Any future schema changes should be added as new migration files (e.g. `002_add_column.up.sql`). These will be applied to all environments, including the imported database. > **Note** > > This approach ensures that new environments get the full schema from scratch via the bootstrap migration, while your production database continues from where it is without any changes being re-applied. --- Source: https://encore.dev/docs/platform/infrastructure/gcp/import-pubsub-topic Section: Deploying # Import an existing Pub/Sub topic > Connecting a Pub/Sub topic to an existing GCP Pub/Sub topic When deploying to your own cloud, the Encore platform provisions the infrastructure backing your Pub/Sub topics automatically. If you already have a GCP Pub/Sub topic, you can instead connect an Encore Pub/Sub topic directly to that existing topic. > **Important** > > The Infrastructure page only lists resources after they've been provisioned. This means importing **replaces** a topic that Encore has already provisioned — deploy your application first so the topic appears, then import it to point at your existing Pub/Sub topic. ## Benefits Using an existing Pub/Sub topic allows you to: - Reuse a topic that is shared with systems outside of Encore - Preserve existing configuration and access policies - Point Encore at a topic you already manage ## Prerequisites - A [connected GCP account](https://encore.dev/docs/platform/infrastructure/gcp) on the environment. - The Encore service account must have permission to access the Pub/Sub topic you want to import. - A Pub/Sub topic declared in your application: ```typescript import { Topic } from "encore.dev/pubsub"; export const signups = new Topic("signups", { deliveryGuarantee: "at-least-once", }); ``` ```go var Signups = pubsub.NewTopic[*SignupEvent]("signups", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) ``` ## Importing a Pub/Sub topic 1. Open the **Infrastructure** page for your environment in the [Encore platform dashboard](https://app.encore.dev). 2. Locate the Pub/Sub topic you want to connect and click **Import**. 3. In the dialog: - Select the **GCP Account** that owns the topic. - Enter the **Project ID** and **Topic ID** of the existing topic. 4. Click **Import**. Importing creates a proposed infrastructure change. Review it in the pending changes summary and click **Deploy** to apply it. On the next deploy, the Encore platform switches your Pub/Sub topic to the existing topic, replacing the topic it previously provisioned. > **Note** > > If the resource can't be resolved, double-check the project and topic ID, and that Encore has permission to view the topic in the selected GCP account. --- Source: https://encore.dev/docs/platform/infrastructure/gcp/import-gcs-bucket Section: Deploying # Import an existing Cloud Storage bucket > Connecting an object storage bucket to an existing GCP Cloud Storage bucket When deploying to your own cloud, the Encore platform provisions the infrastructure backing your object storage buckets automatically. If you already have a GCP Cloud Storage bucket, you can instead connect an Encore bucket directly to that existing bucket. > **Important** > > The Infrastructure page only lists resources after they've been provisioned. This means importing **replaces** a bucket that Encore has already provisioned — deploy your application first so the bucket appears, then import it to point at your existing Cloud Storage bucket. ## Benefits Using an existing Cloud Storage bucket allows you to: - Reuse a bucket and its existing objects without migrating data - Keep buckets that are shared with systems outside of Encore - Preserve existing configuration and access policies ## Prerequisites - A [connected GCP account](https://encore.dev/docs/platform/infrastructure/gcp) on the environment. - The Encore service account must have permission to access the bucket you want to import. - A bucket declared in your application: ```typescript import { Bucket } from "encore.dev/storage/objects"; export const profilePictures = new Bucket("profile-pictures"); ``` ```go var ProfilePictures = objects.NewBucket("profile-pictures", objects.BucketConfig{}) ``` ## Importing a Cloud Storage bucket 1. Open the **Infrastructure** page for your environment in the [Encore platform dashboard](https://app.encore.dev). 2. Locate the bucket resource, expand it, and click **Import** next to the bucket you want to connect. 3. In the dialog: - Select the **GCP Account** that owns the bucket. - Enter the **Project ID** and **Bucket Name** of the existing bucket. 4. Click **Import**. Importing creates a proposed infrastructure change. Review it in the pending changes summary and click **Deploy** to apply it. On the next deploy, the Encore platform switches your bucket to the existing Cloud Storage bucket, replacing the bucket it previously provisioned. > **Note** > > If the resource can't be resolved, double-check the project and bucket name, and that Encore has permission to view the bucket in the selected GCP account. --- Source: https://encore.dev/docs/platform/infrastructure/gcp/import-project Section: Deploying # Import an existing GCP project > Using your pre-existing GCP project instead of provisioning a new one When deploying applications to your own cloud, the Encore platform can provision all necessary infrastructure—including new GCP projects. However, if you already have a GCP project, you can deploy your Encore application directly to this existing project. ## Benefits Using an existing GCP project allows you to: - Keep all your infrastructure in a single project - Maintain existing IAM policies and permissions - Utilize existing billing settings and quotas - Consolidate resources for easier management ## Importing a GCP project Follow these steps to import your existing GCP project: 1. Navigate to **Create Environment** in the [Encore platform dashboard](https://app.encore.dev) 2. Select the GCP cloud provider 3. Choose **Import Project** 4. Add permissions for the Encore Service Account: - Copy the `Encore GCP Service Account` from the cloud dashboard - Go to your project's IAM page in the GCP Console - Grant the `Owner` role to the `Encore GCP Service Account` 5. Return to the Encore platform dashboard 6. Enter your `Project ID` 7. Click the `Resolve` button to validate the project Once validated, you can create the environment. When you deploy to this environment, the Encore platform will automatically deploy your application to your imported GCP project rather than provisioning a new one. --- Source: https://encore.dev/docs/platform/infrastructure/configure-network Section: Deploying # Configure network settings > Customizing IP ranges for network peering When deploying applications with the Encore platform, a network is automatically provisioned with default settings. However, if you plan to peer your Encore network with an existing network, you can manually configure the IP range for your environment. ## Benefits Configuring custom network settings allows you to: - Connect your Encore application to existing networks via peering - Prevent IP range conflicts with other networks in your organization - Plan your network topology with predictable addressing ## Configuring network settings Follow these steps to configure custom network settings: 1. Navigate to **Create Environment** in the Encore platform dashboard 2. Select the AWS or GCP cloud provider 3. Expand the **Network** section 4. Enter your desired IP range - The range must be at least a /16 block to reserve enough IPs for your application to grow - Choose a range that doesn't conflict with your existing networks Once configured, Encore will use your specified IP range instead of assigning a random private network. ## Default network behavior By default, Encore will reserve a randomly assigned /16 block in one of the private IP ranges. This is suitable for most deployments that don't require network peering. --- Source: https://encore.dev/docs/platform/infrastructure/gcp/static-ip Section: Deploying # Configure a static outbound IP > Giving your Cloud Run services a stable egress IP address By default, Cloud Run services send outbound traffic to the internet using a pool of Google-owned IP addresses that changes over time. This makes it impossible for a third party to allowlist your application's traffic. To get a stable outbound IP address, you route your services' egress traffic through your environment's VPC and out via a [Cloud NAT][gcp-cloud-nat] gateway that uses a reserved static IP address. This requires two steps: 1. Create a Cloud NAT gateway with a reserved static IP for the Encore VPC, using the GCP Console. 2. Configure your Cloud Run services to route **All traffic** through the VPC, using the Encore platform dashboard. ## Benefits A static outbound IP allows you to: - Have partners and third-party APIs allowlist your application's IP address - Connect to external databases and services that require IP-based access control - Meet compliance requirements that mandate known, auditable egress addresses ## Step 1: Create a Cloud NAT gateway The Encore platform provisions a dedicated GCP project and VPC network for each environment. The Cloud NAT gateway needs to be created in that project, in the same region as your environment. First, reserve a static external IP address: 1. In the [GCP Console](https://console.cloud.google.com), select the GCP project for your Encore environment 2. Go to **VPC network > IP addresses** and click **Reserve external static IP address** 3. Give it a name (e.g. `encore-egress-ip`) 4. Set **Type** to `Regional` and select the region your environment is deployed in 5. Click **Reserve** Then create the Cloud NAT gateway: 1. Go to **Network services > Cloud NAT** and click **Get started** / **Create Cloud NAT gateway** 2. Give the gateway a name (e.g. `encore-egress-nat`) 3. Set **NAT type** to `Public` 4. Select the VPC **Network** that the Encore platform provisioned for the environment, and the same **Region** as above 5. For **Cloud Router**, create a new router (or select an existing one in the same network and region) 6. Under **Cloud NAT mapping**, keep the default source selection so that all subnet IP ranges are able to use the gateway 7. Set **NAT IP addresses** to `Manual` and select the static IP address you reserved 8. Click **Create** > **Note** > > Create the Cloud NAT gateway *before* changing the VPC egress setting in the next step. Once egress is routed through the VPC, outbound internet traffic depends on the NAT gateway — without it, your services will not be able to reach the internet. ## Step 2: Route all Cloud Run traffic through the VPC By default, Cloud Run only sends traffic destined for private IP ranges through the VPC, and sends internet-bound traffic directly out via Google's IP pool. To make internet-bound traffic use the NAT gateway, you need to change the VPC egress setting: 1. Open your environment in the [Encore platform dashboard](https://app.encore.dev) 2. Go to the **Infrastructure** tab 3. Find the Cloud Run configuration and set **VPC egress** to `All traffic` 4. Apply the change Infrastructure changes made in the dashboard are applied as part of a deployment, so trigger a new deploy (or push a commit) for the setting to take effect. See [Managing Infrastructure](https://encore.dev/docs/platform/infrastructure/managing-infrastructure#deployment-phases) for details on the deployment phases. ## Verifying the configuration Once deployed, make an outbound request from one of your services to a service that echoes back the caller's IP address (for example `https://api.ipify.org`) and confirm that the returned address matches the static IP you reserved. ## Things to keep in mind - **All outbound traffic is affected.** With `All traffic` egress, every outbound request — including calls to public APIs — goes through the VPC and the NAT gateway. This adds NAT data processing charges, and the NAT gateway becomes a dependency for all internet access from your services. - **One gateway per region and network.** Cloud NAT is regional. If an environment spans multiple regions, create a gateway with a reserved IP in each region. - **Multiple IPs for high volume.** A single NAT IP supports a limited number of concurrent connections to the same destination. If your application makes a large number of concurrent outbound connections, assign several reserved static IPs to the gateway and allowlist all of them. - **Manual changes are preserved.** The Encore platform does not overwrite resources it doesn't manage, so the Cloud NAT gateway and reserved IP you create in the console will be left untouched by future deployments. See [Infrastructure Configuration](https://encore.dev/docs/platform/infrastructure/configuration#manual-configuration-in-your-cloud-providers-console). > **Note** > > If your environment runs on GKE instead of Cloud Run, the VPC egress setting doesn't apply — pods already egress through the VPC network. In that case, check whether a Cloud NAT gateway already exists for the network and attach a reserved static IP address to it rather than creating a second gateway. [gcp-cloud-nat]: https://cloud.google.com/nat/docs/overview --- Source: https://encore.dev/docs/platform/infrastructure/aws Section: Deploying # AWS Infrastructure > Understanding your application's AWS infrastructure The Encore platform simplifies the process of deploying applications by automatically provisioning and managing the necessary AWS infrastructure. This guide provides a detailed look at the components involved and how they work together to support your applications. ## Core Infrastructure Components ### Networking Architecture Networking is a critical aspect of cloud infrastructure, ensuring secure and efficient communication between different parts of your application. The Encore platform creates an isolated [Virtual Private Cloud (VPC)][aws-vpc] for each environment, which serves as a secure network boundary. The network architecture is designed with reliability and security in mind. Each VPC spans across two Availability Zones (AZs), providing redundancy and fault tolerance. If one AZ experiences issues, your application can continue running in the other AZ, significantly reducing the risk of downtime. This multi-AZ setup is crucial for maintaining high availability in production environments. Within the VPC, the Encore platform implements a three-tier architecture that carefully separates different components of your application into distinct subnet layers. This separation of concerns enhances both security and performance by controlling traffic flow between layers and limiting potential attack vectors. Each tier is configured with specific security groups and network ACLs to enforce these boundaries, creating a robust and secure networking foundation for your application. #### Subnet Tiers 1. **Public Subnet** The public subnet contains several key components that manage external traffic flow. At the forefront is the Application Load Balancer (ALB), which serves as the entry point for all incoming traffic to your application. The ALB intelligently distributes requests across your application instances, ensuring optimal performance and reliability. To enable outbound communication, the subnet includes an Internet Gateway that allows your application components to securely connect to external services and APIs. Working alongside it is a NAT Gateway, which provides a secure pathway for resources in private subnets (like your compute instances) to access the internet while remaining protected from direct external access. This NAT Gateway acts as an intermediary, translating private IP addresses to public ones for outbound traffic while maintaining the security of your internal resources. 2. **Compute Subnet** The compute subnet is where your application's containers run, regardless of whether you're using Fargate or EKS as your container orchestration platform. This subnet is carefully isolated and configured to only accept incoming traffic from the Application Load Balancer in the public subnet. This strict traffic control ensures that your application containers can only be accessed through proper channels, protecting them from unauthorized direct access while still allowing legitimate requests to flow through seamlessly. 3. **Storage Subnet** (provisioned as needed) The storage subnet is a dedicated network segment designed to host your application's databases and caching systems. To maintain the highest level of security, this subnet operates in complete isolation from the internet, with no direct inbound or outbound connectivity. Access to resources within the storage subnet is strictly limited to traffic originating from the compute subnet, creating a secure enclave for your data layer. This architecture ensures that your sensitive data remains protected while still being readily accessible to your application's services running in the compute tier. ### Container Management The Encore platform provisions an [Elastic Container Registry (ECR)][aws-ecr] to store your application's Docker images. The registry is seamlessly integrated with your chosen compute platform and provides robust security features. Access to images is tightly controlled through comprehensive access controls, ensuring only authorized users and services can pull or push container images. Additionally, ECR automatically scans all images for known security vulnerabilities as they are pushed to the registry, helping you maintain a secure application environment by identifying potential risks before deployment. ### Secrets Management Managing sensitive information securely is crucial. The Encore platform uses [AWS Secrets Manager][aws-secrets] to store and manage secrets, such as API keys and database credentials. Through deep integration with AWS Secrets Manager, the Encore platform automatically injects secrets directly into your service's environment variables at runtime, making them easily accessible while maintaining strict security controls. All secrets are encrypted both at rest and in transit using industry-standard encryption algorithms, providing comprehensive protection for your sensitive data. The system implements fine-grained access controls, where each service is given precisely scoped permissions to access only the specific secrets it needs. This ensures that even if one service is compromised, the blast radius is contained and other secrets remain secure. ## Compute Options The Encore platform provisions one of two compute platforms for running your application containers, based on your choice: ### AWS Fargate When using Fargate, the Encore platform configures: - **Task Definitions** Task definitions are meticulously configured to ensure optimal performance and reliability of your services. Each service's container settings are fine-tuned based on its specific requirements, including memory allocation, CPU utilization, and networking parameters. Comprehensive health check configurations monitor the service's status, enabling quick detection and recovery from any issues. Environment variables are securely injected from AWS Secrets Manager at runtime, providing your services with the credentials and configuration they need while maintaining security. The task definitions are also integrated with AWS Service Discovery, enabling automatic service registration and allowing for seamless service-to-service communication within your application. - **Fargate Services** Fargate services are configured with sophisticated deployment strategies that ensure zero downtime during updates. When deploying new versions of your services, the Encore platform orchestrates a rolling update process where new tasks are gradually introduced while old ones are removed, maintaining consistent availability throughout the deployment. Each service is automatically integrated with Application Load Balancer target groups, enabling intelligent request routing and load distribution. The load balancer continuously monitors the health of your service instances and automatically routes traffic only to healthy targets. To ensure smooth service startup, appropriate health check grace periods are configured. This gives your services adequate time to initialize and warm up before receiving traffic, preventing premature health check failures during deployment or scaling events. - **IAM Configuration** The Encore platform implements a comprehensive IAM security model by creating unique execution roles for each task definition. These roles are automatically configured with precisely scoped permissions that enable secure access to required AWS services. The execution roles allow containers to pull images from ECR and write operational logs to CloudWatch for monitoring and debugging. They also grant access to assigned AWS resources like S3 buckets and SQS queues that the service needs to interact with. Additionally, the roles are configured to securely retrieve secrets from AWS Secrets Manager at runtime, enabling safe storage and access of sensitive configuration data. This granular permission model follows security best practices by providing each service with the minimum privileges required for operation. - **Network Integration** Fargate tasks are strategically placed within private compute subnets, ensuring they remain isolated from direct internet access while maintaining the ability to communicate with other application components. The associated security groups are configured with precise rules that govern network traffic. These rules allow inbound traffic exclusively from the Application Load Balancer, ensuring that your services can only be accessed through the properly configured entry point. For outbound connectivity, the security groups permit traffic to flow to your databases and caching layers, enabling your services to interact with these essential backend resources while maintaining a secure network boundary. ### Amazon EKS When using EKS, the Encore platform configures: - **Compute** Your services run on AWS Fargate, on an EKS managed node group of EC2 instances, or on nodes managed by EKS Auto Mode, as chosen when you create the environment. See [EKS compute](https://encore.dev/docs/platform/infrastructure/kubernetes#eks-compute) for how they differ. You can also [import an existing EKS cluster](https://encore.dev/docs/platform/infrastructure/import-kubernetes-cluster#importing-an-eks-cluster) instead of having Encore create one. - **Cluster Setup** The Encore platform configures the core networking components required for cluster operation. The VPC CNI (Container Network Interface) is configured to enable pod networking within the cluster, allowing pods to communicate efficiently using the underlying AWS VPC networking capabilities. This includes setting up IP address management and network policy enforcement. The cluster's internal DNS resolution is handled through CoreDNS, which is configured for optimal service discovery and name resolution within the cluster. CoreDNS settings are tuned to provide fast and reliable DNS lookups while maintaining reasonable cache sizes and query limits. - **Kubernetes Resources** The Encore platform automatically manages all necessary Kubernetes resources for your application. Each service in your application is deployed as a separate Kubernetes Deployment, allowing for independent scaling and lifecycle management. These deployments are configured with appropriate resource requests, limits, and health checks to ensure reliable operation. For authentication and authorization, the Encore platform implements IAM Roles for Service Accounts (IRSA), providing secure access to AWS services. Each service gets its own service account with precisely scoped IAM roles, following the principle of least privilege. For sensitive data like API keys and credentials, the Encore platform uses Kubernetes Secrets, which are encrypted at rest and only accessible to authorized services. To enable network connectivity, the Encore platform creates Kubernetes Service resources for each of your application's services, providing stable networking endpoints for inter-service communication. - **Load Balancer Integration** The Encore platform manages the complete load balancer integration for your EKS cluster. The AWS Load Balancer Controller is automatically installed and configured to handle ingress traffic for your application. On EKS Auto Mode, the load balancing built into Auto Mode takes its place. This controller works in conjunction with the Application Load Balancer (ALB) to provide intelligent traffic routing and SSL/TLS termination. The ALB Ingress Controller is configured to automatically create and manage Application Load Balancers based on your application's needs. It handles the creation and configuration of target groups, ensuring traffic is properly distributed across your service pods. The controller also manages the lifecycle of these resources, automatically cleaning up unused resources to prevent waste. Target group binding is automatically configured to map your Kubernetes services to the appropriate ALB target groups. This ensures that traffic is correctly routed to the right pods and that health checks are properly configured to maintain high availability. For secure communication, the Encore platform automatically manages SSL/TLS certificates through AWS Certificate Manager. These certificates are automatically provisioned, renewed, and attached to your load balancers, ensuring all external traffic to your application is encrypted. The system also handles certificate rotation and updates transparently, maintaining secure communication without manual intervention. - **Monitoring Setup** The Encore platform automatically aggregates and sends metrics to your configured metrics destination, providing you with real-time visibility into your application's performance. In addition to metrics, the Encore platform configures the CloudWatch Logs agent to capture and forward all container logs. The logs are structured and organized by service name, making it easy to search and analyze application behavior. Log streams are automatically created for each container, and log retention policies are configured to help manage storage costs while maintaining necessary historical data. - **Service Accounts** The Encore platform implements a comprehensive service account management system that ensures secure and controlled access to resources. Each service in your application receives its own dedicated Kubernetes service account, providing a unique identity for authentication and authorization purposes. To enable secure interaction with AWS services, the Encore platform maps each Kubernetes service account to a corresponding IAM role using IAM Roles for Service Accounts (IRSA). This mapping allows pods to securely authenticate with AWS services without storing long-lived credentials. The IAM roles are automatically configured with the minimum required permissions for each service's needs. This includes access to service-specific S3 buckets for object storage operations, permissions to publish and subscribe to SQS queues and SNS topics, ability to retrieve secrets from AWS Secrets Manager, and secure access to assigned database instances. These permissions are continuously updated as your application evolves, ensuring services always have the access they need while maintaining strong security boundaries. All of these configurations are automatically maintained and updated by the Encore platform as you develop your application, ensuring your infrastructure stays aligned with your application's needs. ## Managed Services ### Databases The Encore platform runs your PostgreSQL databases on Amazon RDS. When you create an environment, you choose how it gets its database server in the **Database** section: - **Amazon RDS Postgres** (the default): a standalone [RDS for PostgreSQL][aws-rds] DB instance. - **Amazon Aurora Postgres**: an [Aurora PostgreSQL][aws-aurora] DB cluster. Its storage grows on its own and is replicated across Availability Zones, and you can add read replicas and use Aurora Serverless v2. It costs more than RDS at rest. - **Import Existing Database**: an RDS for PostgreSQL instance or Aurora PostgreSQL cluster you already have. See [Import an existing RDS or Aurora database](https://encore.dev/docs/platform/infrastructure/aws/import-rds). - **Neon database**: a [Neon](https://encore.dev/docs/platform/infrastructure/neon) Postgres database instead of RDS. The choice is made when the environment is created and can't be changed afterwards. All the databases your application declares live on the environment's one database server, each as its own logical database. #### Amazon RDS for PostgreSQL A new RDS instance comes up with: | Setting | Default | | --- | --- | | PostgreSQL version | 18.3 | | Instance class | `db.m5.large` | | Storage | 10 GiB of `gp2`, autoscaling up to 20 GiB | | Multi-AZ | Enabled | | Backup retention | 7 days | | Max connections | 1000 | #### Amazon Aurora PostgreSQL A new Aurora cluster comes up with: | Setting | Default | | --- | --- | | PostgreSQL version | Aurora PostgreSQL 18.3 | | Instances | 1 writer on `db.r6g.large` | | Storage type | `aurora` (Aurora Standard) | | Backup retention | 7 days | | Max connections | 1000 | You can add up to 15 readers, which start out with the writer's instance class, and pick a storage type of `aurora-iopt1` (I/O-Optimized) to avoid per-I/O charges. To use [Aurora Serverless v2][aws-aurora-serverless], set an instance's class to `db.serverless` and set the cluster's serverless capacity range, between 0 and 256 ACUs in steps of 0.5. The range is shared by every `db.serverless` instance in the cluster, and a minimum of 0 lets them pause when idle. #### Configuration Both kinds of database server are encrypted at rest, have deletion protection enabled, and have Performance Insights enabled with the free 7-day retention. Everything else in the tables above can be changed in the [Encore Cloud dashboard](https://app.encore.dev) under **(Select your environment) > Infrastructure**, along with the CA certificate and, on RDS, the provisioned IOPS. On Aurora the instance class and CA certificate are set per instance and everything else on the cluster. Changing the max connections or the PostgreSQL version reboots the instances, and a major version upgrade moves them onto a parameter group for the new version. PostgreSQL versions can only be upgraded, never downgraded. #### Database Access Database servers are placed in the private storage subnets, with no direct internet access, and only accept connections from your application's compute subnets. Each service in your application gets its own database role, with access to only the databases it uses. Migrations and the Encore platform's own access go through [Emissary](https://github.com/encoredev/emissary), a secure SOCKS proxy running in your environment, so the database never has to be exposed. #### Database Roles The Encore platform sets up a hierarchy of PostgreSQL roles on the database server, so that each part of your application gets only the access it needs: | Role | Purpose | | --- | --- | | `encore_platform` | The Encore platform's own role. It has `CREATEDB` and `CREATEROLE` and is used to create and manage everything below. The Encore platform connects as the login role `encore_platform_`, which it creates through the master user and which can act as `encore_platform`. | | `db__reader` | Can connect to the database and read every table and sequence in every schema. | | `db__writer` | Everything the reader can do, plus insert, update and delete rows and execute functions and procedures. | | `db__admin` | Everything the writer can do, plus all other privileges on the database and its schemas, tables, sequences and routines. It owns the databases the Encore platform creates, and your migrations run as it. | | `svc_` | One per service. A member of `db__writer` for each database the service uses. | | `encore_container_` | A login role per container (Fargate task or Kubernetes deployment), a member of the `svc_` roles of the services running in it. These are the credentials your application connects with. | | `encore_reader`, `encore_writer`, `encore_admin` | Members of the reader, writer and admin role of every database, with a login role each (`encore_reader_` and so on). They're used to connect at that access level through the Encore platform, such as with [`encore db shell`](https://encore.dev/docs/ts/primitives/databases#connecting-to-databases). | | `encore_services` | Groups every `svc_` role. | Grants on tables, sequences and routines are made in every schema, and default privileges are set for the role your migrations run as, so tables your migrations create later are covered too. Each container has its own credentials, which you can view and rotate as described in [Managing database user credentials](https://encore.dev/docs/platform/infrastructure/manage-db-users). ### Pub/Sub The Encore platform implements a robust messaging system using [SQS][aws-sqs] and [SNS][aws-sns]. The system automatically configures dead-letter queues to capture failed messages, enabling thorough analysis and debugging of messaging issues. Each service in your application receives precisely scoped IAM permissions to publish and consume messages, ensuring secure communication between components. The Encore platform fully manages the creation and configuration of subscriptions and topics, streamlining the setup and ongoing maintenance of your messaging infrastructure while maintaining optimal performance and reliability. ### Object Storage The Encore platform leverages [S3][aws-s3] for object storage, providing a comprehensive solution for your application's storage needs. When you declare storage requirements in your application, the Encore platform automatically provisions dedicated S3 buckets with unique names to ensure global uniqueness across AWS. Each service in your application receives precisely scoped permissions to perform storage operations, following the principle of least privilege. For public buckets, the Encore platform automatically integrates with CloudFront to create a global content delivery network, significantly improving access speeds for your users worldwide. Each bucket is assigned its own unique domain name, making it simple to manage and access stored content while maintaining a clear organizational structure. ### Caching The Encore platform uses [ElastiCache for Redis][aws-redis] to provide a high-performance caching solution. The service starts with cache.m6g.large instances that can automatically scale up as your application's needs grow. To ensure maximum reliability, caches are configured with Multi-AZ replication across availability zones, providing both high availability and fault tolerance. In the event of any failures, automatic failover capabilities ensure your application experiences no disruption in service. Security is maintained through Redis Access Control Lists (ACLs), which provide fine-grained control over who can access your cache and what operations they can perform. The entire system is configured for high availability, with monitoring and alerting in place to maintain optimal performance and uptime. This comprehensive setup ensures your application's caching layer remains fast, secure, and always available. ### Cron Jobs The Encore platform provides a streamlined approach to scheduled tasks that prioritizes security and simplicity. Each cron job is executed through authenticated API requests that are cryptographically signed to verify their authenticity. The system performs rigorous source verification to ensure all scheduled tasks originate exclusively from the Encore platform's cron functionality, preventing unauthorized execution attempts. This elegant implementation requires no additional infrastructure components, making it both cost-effective and easy to maintain while ensuring your scheduled tasks run reliably and securely. [aws-vpc]: https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html [aws-fargate]: https://aws.amazon.com/fargate/ [aws-eks]: https://aws.amazon.com/eks/ [aws-secrets]: https://aws.amazon.com/secrets-manager/ [aws-rds]: https://aws.amazon.com/rds/postgresql/ [aws-aurora]: https://aws.amazon.com/rds/aurora/ [aws-aurora-serverless]: https://aws.amazon.com/rds/aurora/serverless/ [aws-sqs]: https://aws.amazon.com/sqs/ [aws-sns]: https://aws.amazon.com/sns/ [aws-s3]: https://aws.amazon.com/s3/ [aws-redis]: https://aws.amazon.com/elasticache/redis/ [aws-ecr]: https://aws.amazon.com/ecr/ [aws-alb]: https://aws.amazon.com/elasticloadbalancing/application-load-balancer/ --- Source: https://encore.dev/docs/platform/infrastructure/aws/import-rds Section: Deploying # Import an existing RDS or Aurora database > Using your pre-existing database instead of provisioning a new one When deploying applications to your own cloud, the Encore platform can provision all necessary infrastructure, including the database server. However, if you already have an Amazon RDS for PostgreSQL DB instance or an Amazon Aurora PostgreSQL DB cluster, you can connect your Encore application directly to it instead. ## Benefits Using an existing database allows you to: - Maintain data continuity with your existing systems - Preserve specific database configurations - Utilize familiar database setups without migration ## Requirements The database server you import must: - **Run PostgreSQL.** It must be an RDS for PostgreSQL DB instance or an Aurora PostgreSQL DB cluster. Other RDS engines, such as MySQL, can't be imported. - **Have its master password managed by AWS Secrets Manager.** Encore reads the master password from the secret RDS manages for it. If your database doesn't use this yet, modify it in the RDS console and enable **Manage master credentials in AWS Secrets Manager**. Doing so changes the master password, so update anything else that uses it. - **Be in a VPC.** Encore reaches the database through its DB subnet group and security group. Aurora clusters created with *express configuration* aren't in a VPC and can't be moved into one. To import one, restore a snapshot of it as a cluster with full configuration in a VPC, with the master password managed by Secrets Manager, and import the restored cluster. - **Be in the AWS account and region of the environment.** The Encore IAM role must be allowed to describe it (`rds:DescribeDBInstances`, and `rds:DescribeDBClusters` for an Aurora cluster) and read its master user secret. For an Aurora cluster, import the cluster itself rather than one of its DB instances. ## Importing a database You import a database when you create an environment: 1. Navigate to **Create Environment** in the [Encore platform dashboard](https://app.encore.dev) 2. Select the AWS cloud provider 3. Pick the `AWS Region` your database resides in 4. In the **Database** section, choose **Import Existing Database** 5. Enter the identifier of your RDS DB instance or Aurora DB cluster, as shown in the RDS console 6. Click the `Resolve` button to validate it If the database can't be imported, the dashboard tells you which requirement it doesn't meet, with a link to it in the RDS console. Once it's resolved, you can create the environment. When you deploy to this environment, the Encore platform connects your application to the imported database rather than provisioning a new one. ## What Encore manages An imported database server remains yours. Encore doesn't change its settings and never deletes it, nor its subnet group, security group, or parameter groups. For an Aurora cluster, Encore keeps the DB instances the cluster already has. To connect your application to it, Encore: - Adds an inbound rule to the database's security group allowing your services to connect on the database port, and the network ACL rules to go with it. - Sets up VPC peering between the environment's VPC and the database's VPC, if they're different. The two VPCs' IP ranges must not overlap, so pick a free range for the environment's VPC when you create it if needed. - Runs [Emissary](https://github.com/encoredev/emissary), a secure SOCKS proxy, in the environment's compute, and allows it to connect to the database the same way as your services. The Encore platform reaches the database through Emissary to create roles and databases and run your migrations, so the database doesn't have to be reachable from the internet. - Creates the [PostgreSQL roles](#roles) it needs, using the master user, and a database for each database your application declares that doesn't exist yet. When you delete the environment, Encore removes the roles, grants, and databases it created. Anything that was there before stays. ## Mapping existing databases to your Encore app To access an existing database in your Encore application, you need to specify the name of the existing database when you declare the database in your app. For example, if you have an existing database called `mydb` you can create a reference to it like so: ```typescript const db = new SQLDatabase("mydb"); ``` ```go sqldb.NewDatabase("mydb", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` Encore leaves databases, roles, role memberships, and grants that already exist on the server alone: it uses them, but never modifies, revokes, or deletes them. ### Roles Encore sets up the same [role hierarchy](https://encore.dev/docs/platform/infrastructure/aws#database-roles) on an imported server as on one it creates: `encore_platform` for itself, a reader, writer and admin role per database, a role per service, and a login role per container. It creates them through the master user, which ends up a member of `encore_platform`. The roles your application connects with only get access to the databases it declares, never to the master user or to other databases on the server. If a role with one of these names already exists and can't log in, Encore uses it as it is and won't change or drop it. Your own roles are otherwise left alone, and none of them is granted to Encore's roles, except a database's owner as described below. ### Table ownership An existing database's tables are usually owned by the role that created them, often the master user, and not by Encore. So that your services can read and write them and your migrations can alter them, Encore runs your migrations as the database's owner, and grants your services access to the tables in every schema on each deploy: - If the database is owned by the master user, Encore connects as the master user to run migrations and apply grants. - If it's owned by another role, Encore grants that role to its own platform role, and revokes it again when the environment is deleted. Encore never grants the owner to the roles your application connects with. - If it's owned by a superuser, Encore leaves it alone. Grant your services access yourself. Owning a database doesn't by itself allow a role to alter tables, or grant access to them, if another role owns those tables. Only a table's owner, a member of the owner role, or a superuser can do that. So before you deploy, check who owns the tables your migrations alter and your services use (for example with `\dt *.*` in `psql`). If any are owned by a role other than the database's owner, make the database's owner a member of that role, or change the tables' owner to the database's owner. Otherwise those migrations and grants will fail. ## Applying migrations to existing databases Encore uses a table called `schema_migrations` in the public namespace to keep track of which migrations have been applied. If you import an existing database without that table, Encore will create it for you and apply your migrations in order. If the table already exists, Encore expects it to contain exactly two columns: ``` version bigint dirty boolean ``` If the table exists but has a different schema, you will not be able to import it with Encore at this time. If the table exists with an existing entry, Encore will apply all higher versions in your `migrations` directory to the database. ### Bootstrap migration for existing databases If you have an existing production database that wasn't previously managed by Encore's migration system, you need to set up a **bootstrap migration** so that new environments (dev, staging, etc.) are created with the same schema, while your existing database is not modified. **Step 1:** Dump your existing database schema: ```shell $ pg_dump --schema-only --no-owner --no-privileges your_database > 001_bootstrap.up.sql ``` Review the output and clean it up as needed (remove any database-specific settings, comments, etc.), then place it in your service's `migrations` directory. **Step 2:** In your **existing** imported database, manually create the `schema_migrations` table and mark the bootstrap migration as already applied: ```sql CREATE TABLE IF NOT EXISTS schema_migrations (version bigint NOT NULL, dirty boolean NOT NULL); INSERT INTO schema_migrations (version, dirty) VALUES (1, false); ``` This tells Encore that migration `1` (the bootstrap) has already been applied to this database. Encore will skip it and only run migrations with a higher version number. **Step 3:** Any future schema changes should be added as new migration files (e.g. `002_add_column.up.sql`). These will be applied to all environments, including the imported database. > **Note** > > This approach ensures that new environments get the full schema from scratch via the bootstrap migration, while your production database continues from where it is without any changes being re-applied. --- Source: https://encore.dev/docs/platform/infrastructure/aws/import-sns-topic Section: Deploying # Import an existing SNS topic > Connecting a Pub/Sub topic to an existing AWS SNS topic When deploying to your own cloud, the Encore platform provisions the infrastructure backing your Pub/Sub topics automatically. If you already have an AWS SNS topic, you can instead connect an Encore Pub/Sub topic directly to that existing topic. > **Important** > > The Infrastructure page only lists resources after they've been provisioned. This means importing **replaces** a topic that Encore has already provisioned — deploy your application first so the topic appears, then import it to point at your existing SNS topic. ## Benefits Using an existing SNS topic allows you to: - Reuse a topic that is shared with systems outside of Encore - Preserve existing configuration and access policies - Point Encore at a topic you already manage ## Prerequisites - A [connected AWS account](https://encore.dev/docs/platform/infrastructure/aws) on the environment. - The Encore IAM role must have permission to access the SNS topic you want to import. - A Pub/Sub topic declared in your application: ```typescript import { Topic } from "encore.dev/pubsub"; export const signups = new Topic("signups", { deliveryGuarantee: "at-least-once", }); ``` ```go var Signups = pubsub.NewTopic[*SignupEvent]("signups", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) ``` ## Importing an SNS topic 1. Open the **Infrastructure** page for your environment in the [Encore platform dashboard](https://app.encore.dev). 2. Locate the Pub/Sub topic you want to connect and click **Import**. 3. In the dialog: - Select the **AWS Account** that owns the topic. - Enter the **Topic ARN** in the form `arn:aws:sns:::`. 4. Click **Import**. The account ID in the ARN must match the account ID of the selected AWS integration. Importing creates a proposed infrastructure change. Review it in the pending changes summary and click **Deploy** to apply it. On the next deploy, the Encore platform switches your Pub/Sub topic to the existing SNS topic, replacing the topic it previously provisioned. > **Note** > > If the resource can't be resolved, double-check that the ARN is correct and that Encore has permission to view the topic in the selected AWS account. --- Source: https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket Section: Deploying # Import an existing S3 bucket > Connecting an object storage bucket to an existing AWS S3 bucket When deploying to your own cloud, the Encore platform provisions the infrastructure backing your object storage buckets automatically. If you already have an AWS S3 bucket, you can instead connect an Encore bucket directly to that existing bucket. > **Important** > > The Infrastructure page only lists resources after they've been provisioned. This means importing **replaces** a bucket that Encore has already provisioned — deploy your application first so the bucket appears, then import it to point at your existing S3 bucket. ## Benefits Using an existing S3 bucket allows you to: - Reuse a bucket and its existing objects without migrating data - Keep buckets that are shared with systems outside of Encore - Preserve existing configuration and access policies ## Prerequisites - A [connected AWS account](https://encore.dev/docs/platform/infrastructure/aws) on the environment. - The Encore IAM role must have permission to access the S3 bucket you want to import. - A bucket declared in your application: ```typescript import { Bucket } from "encore.dev/storage/objects"; export const profilePictures = new Bucket("profile-pictures"); ``` ```go var ProfilePictures = objects.NewBucket("profile-pictures", objects.BucketConfig{}) ``` ## Importing an S3 bucket 1. Open the **Infrastructure** page for your environment in the [Encore platform dashboard](https://app.encore.dev). 2. Locate the bucket resource, expand it, and click **Import** next to the bucket you want to connect. 3. In the dialog: - Select the **AWS Account** that owns the bucket. - Enter the **Region** the bucket resides in. - Enter the **Bucket Name**. 4. Click **Import**. Because S3 ARNs don't include the region, the region is entered separately. The account is taken from the selected AWS integration. Importing creates a proposed infrastructure change. Review it in the pending changes summary and click **Deploy** to apply it. On the next deploy, the Encore platform switches your bucket to the existing S3 bucket, replacing the bucket it previously provisioned. > **Note** > > If the resource can't be resolved, double-check the region and bucket name, and that Encore has permission to view the bucket in the selected AWS account. --- Source: https://encore.dev/docs/platform/infrastructure/kubernetes Section: Deploying # Deploy to a new Kubernetes cluster > Deploying your app to a new Kubernetes cluster The Encore platform gives you flexibility in where you run your applications. You have two options for Kubernetes deployments: 1. **Deploy to a new cluster**: the Encore platform can automatically provision and manage a new Kubernetes cluster in your cloud account on AWS or GCP. 2. **Use an existing cluster**: Deploy to your pre-existing GKE or EKS cluster ([see instructions here](https://encore.dev/docs/platform/infrastructure/import-kubernetes-cluster)) All infrastructure provisioning is automated, and configuration is managed through the [Encore platform Dashboard](https://app.encore.dev), keeping your application code clean and infrastructure-agnostic. ## Deploying to a new Kubernetes cluster **1. Connect your cloud account:** Ensure your cloud account (Google Cloud Platform or AWS) is connected to the Encore platform. ([See docs](https://encore.dev/docs/platform/deploy/own-cloud)) **2. Create environment:** Open your app in the [Encore platform dashboard](https://app.encore.dev) and go to **Environments**, then click on **Create Environment**. Next, select your cloud (AWS or GCP) and then specify Kubernetes as the compute platform. On GCP, the Encore platform deploys to GKE. On AWS, it deploys to EKS, and you choose what your services run on (see [EKS compute](#eks-compute) below). You can also configure if you want to allocate all services in one process or run one process per service. ![Environment Settings](https://encore.dev/assets/docs/k8s-config.jpg) **3. Push your code:** To deploy, commit and push your code to the branch you configured as the deployment trigger. You can also trigger a manual deploy from the Cloud Dashboard by going to the **Environment Overview** page and clicking on **Deploy**. **4. Automatic deployment by the Encore platform:** Once you've triggered the deploy, the Encore platform will automatically provision and deploy the necessary infrastructure on Kubernetes, per your environment configuration in the Cloud Dashboard. You can monitor the status of your deploy and view your environment's details through the Encore platform Dashboard. **5. Accessing your cluster with kubectl:** You can access your cluster using the `kubectl` CLI tool. [See the docs](https://encore.dev/docs/platform/infrastructure/configure-kubectl) for how to do this. ## EKS compute On AWS, you choose what your services' pods run on when you create the environment: | Option | Runs pods on | Good for | |---|---|---| | **Kubernetes: EKS Fargate** | AWS Fargate. No nodes to manage; each pod gets its own isolated compute. | Getting started, and workloads that don't need control over their nodes. | | **Kubernetes: EKS Node Groups** | An [EKS managed node group](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) of EC2 instances, scaled by the Kubernetes Cluster Autoscaler. | Workloads that need more control over their compute, or many small services that pack well onto shared nodes. | | **Kubernetes: EKS Auto Mode** | Nodes that [EKS Auto Mode](https://docs.aws.amazon.com/eks/latest/userguide/automode.html) launches, sizes and patches for you. | EC2 nodes without managing node groups. | The cluster's system components (CoreDNS, and on Fargate and node groups the AWS Load Balancer Controller) run on the same compute as your services. Auto Mode is part of how the cluster is created, so it can't be turned on or off for an existing environment. ### Fargate Fargate sizes and bills each pod by its resource requests, rounded up to the nearest Fargate configuration. Encore gives each service container a request of 1 vCPU and 2 GiB of memory, unless you configure other requests. ### Node groups Encore creates one managed node group of `m7i.xlarge` instances (on-demand, x86). The Cluster Autoscaler adds and removes nodes between the group's minimum and maximum size as your pods need them. Encore keeps the node group's Kubernetes version in step with the cluster's. Each service still gets its own security group, using [security groups for pods](https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html), so network access to your databases and caches stays scoped per service. This limits how many pods fit on a node: an `m7i.xlarge` holds up to 18 pods with a security group. ### Auto Mode Encore creates a node pool and node class of its own for your services, running on-demand x86 instances from the general purpose, compute optimized and memory optimized families. EKS chooses the instance types, scales the nodes and replaces them as needed. EKS Auto Mode also provides the load balancing and DNS, so Encore doesn't install the AWS Load Balancer Controller. EKS Auto Mode doesn't support security groups for pods. Instead, all of the environment's nodes share one security group, and access to your databases and caches is granted to that group. Network access is therefore scoped to the environment rather than to each service. Database credentials are per container, as on every Encore deployment: services that run in the same container share them (see [Database roles](https://encore.dev/docs/platform/infrastructure/aws#database-roles)). ## Infrastructure Overview The Encore platform simplifies the process of deploying applications by automatically provisioning and managing the necessary Kubernetes components. Here's an overview of the components the Encore platform manages and how they work together to support your applications. ### Namespace Management The Encore platform creates a unique namespace for each environment deployed to your Kubernetes cluster, ensuring complete isolation between different environments of your application. ### Secrets Management The Encore platform provides comprehensive secrets management through deep integration with Kubernetes Secrets. Application secrets that you configure in the Encore platform are automatically stored as Kubernetes Secrets and made available to your services at runtime. This includes both application-specific secrets that you define, as well as infrastructure secrets like database credentials that the Encore platform manages automatically. Service accounts are automatically bound to the appropriate secrets they need access to, ensuring each service can only access the secrets it requires. This follows the principle of least privilege and helps maintain a strong security posture. ### Ingress Configuration The Encore platform provisions and manages ingress for your applications through a cloud provider-specific ingress controller. The ingress controller is automatically configured to handle incoming traffic and route it securely to your application's Encore Gateway service. It manages TLS certificates automatically to ensure all traffic is encrypted, and provides fine-grained control over which services are accessible from the public internet. The controller configuration is optimized for your specific cloud provider to ensure the best possible performance and reliability. ## Service Management ### Deployments The Encore platform manages the deployment configuration for each service in your application. Each service is deployed as a separate Kubernetes deployment, allowing for independent scaling and management. The deployment configurations are automatically generated and optimized based on your service's requirements. For each service, the Encore platform configures the pod specifications with appropriate resource requests and limits, health checks, and container settings. Runtime configurations like environment variables and command arguments are automatically set based on your application's needs. The container orchestration is handled seamlessly, with the Encore platform managing pod scheduling, updates, and scaling to ensure your services run reliably and efficiently. ### Network Configuration The Encore platform provides a comprehensive networking setup through Kubernetes Service resources. Each service in your application gets assigned a unique cluster IP address, enabling reliable internal communication between services. This IP allocation works in conjunction with Kubernetes' built-in service discovery mechanism, allowing services to locate and communicate with each other using consistent internal DNS names. The internal service routing ensures that requests are efficiently distributed across all available pods for each service, providing automatic load balancing and failover capabilities. ### Identity and Access The Encore platform provides comprehensive service identity management through Kubernetes service accounts. Each pod is assigned its own dedicated service account, which handles authentication with the Kubernetes API and enables secure access to resources. These service accounts are automatically bound to the specific secrets and permissions required by each service. For cloud provider integration, the Encore platform maps the service accounts to appropriate IAM roles, enabling secure access to cloud resources like databases and object storage. Following the principle of least privilege, the Encore platform configures the minimum required permissions for each service account, ensuring services can only access the resources they explicitly need. All these configurations are automatically maintained and updated by the Encore platform as you develop your application, ensuring your infrastructure stays aligned with your application's needs. --- Source: https://encore.dev/docs/platform/infrastructure/import-kubernetes-cluster Section: Deploying # Import an existing Kubernetes cluster > Deploying to your pre-existing cluster instead of provisioning a new one When you deploy your application to your own cloud, the Encore platform can provision infrastructure for it in many different ways – including setting up a Kubernetes cluster. If you already have a Kubernetes cluster, the Encore platform can deploy your Encore application into this pre-existing cluster. This is often useful if you want to integrate your Encore application with other parts of your system that are not built using Encore. You can import a GKE cluster on GCP, or an EKS cluster on AWS. ## Importing a GKE cluster To import your cluster, go to **Create Environment** in the [Encore platform dashboard](https://app.encore.dev), select **Kubernetes: Existing GKE Cluster** as the compute platform, and then specify your cluster's `Project ID`, `Region`, and `Cluster Name`. When you deploy to this environment, the Encore platform will use your imported cluster as the compute instance. ![Import your existing Kubernetes cluster](https://encore.dev/assets/docs/import-k8s.png) ## Importing an EKS cluster ### Requirements The EKS cluster you import must: - **Be active, and in the AWS account and region of the environment.** - **Use IPv4** for its pods and services. IPv6 clusters can't be imported. - **Have its API server endpoint publicly accessible.** The Encore platform talks to the Kubernetes API from outside your VPC. If that public endpoint uses a CIDR allow list (`publicAccessCidrs`), the list must allow Encore's control plane to reach the API. An allow list limited to office or CI addresses causes Resolve or deploy to fail. - **Allow the Encore IAM role to manage access to it.** Encore needs `eks:UpdateClusterConfig`, `eks:CreateAccessEntry` and `eks:AssociateAccessPolicy` on the cluster, to let its role in (see [Cluster access](#cluster-access)). ### Importing the cluster 1. Navigate to **Create Environment** in the [Encore platform dashboard](https://app.encore.dev) 2. Select the AWS cloud provider 3. Pick the `AWS Region` your cluster runs in 4. In the **Compute** section, choose **Kubernetes: Existing EKS cluster** 5. Enter the name of your EKS cluster 6. Click the `Resolve` button to validate it Once it's resolved, you can create the environment. When you deploy to it, the Encore platform deploys your application into the cluster in a namespace of its own. ### Where your services run Your services run on the compute the cluster already has: its first managed node group, a node pool of Encore's own on an [EKS Auto Mode](https://docs.aws.amazon.com/eks/latest/userguide/automode.html) cluster, or otherwise a Fargate profile that Encore creates for the environment's namespace. AWS accepts only private subnets for a Fargate profile: subnets with no direct route to an internet gateway. A cluster with only public subnets cannot use this fallback. See [EKS compute](https://encore.dev/docs/platform/infrastructure/kubernetes#eks-compute) for how each of them works, and [Fargate profiles](https://docs.aws.amazon.com/eks/latest/userguide/fargate-profile.html) for the subnet rule. ### Cluster access EKS only gives a cluster's creator access to its Kubernetes API, so Encore lets its own IAM role in with an [access entry](https://docs.aws.amazon.com/eks/latest/userguide/access-entries.html) for the role, with the `AmazonEKSClusterAdminPolicy` access policy. If the cluster authenticates with the `aws-auth` ConfigMap only, Encore switches its authentication mode to `API_AND_CONFIG_MAP` first. Everyone the ConfigMap lets in keeps their access. EKS can't switch a cluster back, so the cluster keeps this mode after the environment is deleted. ### Load balancers Your public gateway gets an internet-facing Application Load Balancer in the VPC's public subnets: the ones that route to an internet gateway. AWS requires an Application Load Balancer to have subnets in at least two Availability Zones, so an internet-facing gateway needs public subnets in at least two of them. If the VPC has no public subnets, the gateway is private by default and gets an internal load balancer in the cluster's own subnets. Encore doesn't tag your subnets for load balancer discovery. It names the subnets on the Ingress instead, one per availability zone. EKS Auto Mode still requires role tags on the subnets its load balancers use. Add `kubernetes.io/role/elb` (value `1` or empty) on subnets for an internet-facing load balancer, and `kubernetes.io/role/internal-elb` (value `1` or empty) on subnets for an internal one. Encore leaves imported subnets untagged, so apply these tags yourself. See [Tag subnets for EKS Auto Mode](https://docs.aws.amazon.com/eks/latest/userguide/tag-subnets-auto.html). ### What Encore manages An imported cluster remains yours. Encore never deletes the cluster, its VPC and subnets, its IAM roles, or its node groups, and doesn't upgrade the cluster or its node groups. Encore adds what your application needs: - The environment's namespace and everything in it, and the security groups for your services. - On clusters that don't use Auto Mode: the AWS Load Balancer Controller, and with node groups the Cluster Autoscaler, unless they're already installed under the names Encore uses. Ones already installed are left as they are. - On clusters that don't use Auto Mode: [security groups for pods](https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html), which means turning on pod ENIs in the VPC CNI and, with node groups, attaching the `AmazonEKSVPCResourceController` policy to the cluster's IAM role unless it's attached already. - The access entry and authentication mode described in [Cluster access](#cluster-access). When you delete the environment, Encore removes what it added, except the authentication mode and the VPC CNI settings. Anything that was there before stays, including an access entry Encore's role already had. Databases and caches that Encore creates for the environment live in the environment's own VPC, which Encore peers with the cluster's VPC. The two VPCs' IP ranges must not overlap, so pick a free range for the environment's VPC when you create it if needed. --- Source: https://encore.dev/docs/platform/infrastructure/configure-kubectl Section: Deploying # Configure kubectl The Encore platform automatically provisions and manages Kubernetes clusters for you, but sometimes it's useful to manually inspect clusters using the [kubectl](https://kubernetes.io/docs/reference/kubectl/) cli. To do this, you need to configure `kubectl` to connect and authenticate through encore. You can do this by running the following command in your app directory: ```shell encore kubernetes configure -e ``` Where `` is the name of the environment you want to configure `kubectl` for. This will configure `kubectl` to use `encore` to authenticate the cluster and proxy your traffic to the correct cluster. You can now use `kubectl` as you normally would, for example: ```shell kubectl get pods ``` --- Source: https://encore.dev/docs/self-host/build Section: Deploying # Build Docker Images Encore supports building Docker images directly from the CLI, which can then be self-hosted on your own infrastructure of choice. This can be a good choice if [the Encore platform](https://encore.dev/docs/platform/introduction) isn't a good fit for your use case, or if you want to [migrate away](https://encore.dev/docs/migrate-away). ## Building your own Docker image To build your own Docker image, use `encore build docker MY-IMAGE:TAG` from the CLI. This will compile your application using the host machine and then produce a Docker image containing the compiled application. The base image defaults to `scratch` for GO apps and `node:slim` for TS, but can be customized with `--base`. This is exactly the same code path that Encore's CI system uses to build Docker images, ensuring compatibility. By default, all your services are included and started by the Docker image. If you want to specify specific services and gateways to include, you can use the `--services` and `--gateways` flags. ```bash encore build docker --services=service1,service2 --gateways=api-gateway MY-IMAGE:TAG ``` You can target a specific architecture with `--arch` (useful when your build machine differs from your deploy target): ```bash encore build docker --arch=arm64 MY-IMAGE:TAG ``` To provide an [infrastructure configuration](https://encore.dev/docs/self-host/configure-infra) file at build time, use `--config`. This is required if your app uses databases, caches, Pub/Sub, object storage, or secrets, otherwise the build fails with an incomplete configuration error: ```bash encore build docker --config=infra-config.json MY-IMAGE:TAG ``` The image runs your services, but it doesn't run database migrations or trigger cron jobs: - **Database migrations:** Apply your migrations to each database before starting a new image, for example with [golang-migrate](https://github.com/golang-migrate/migrate), which uses the same migration file naming as Encore. - **Cron jobs:** Call each cron job's endpoint on its schedule from your own scheduler. `encore build docker` prints the endpoint path and schedule of each cron job. Private endpoints can't be called from outside your app, so the endpoint must be public and require authentication; ensure that your scheduler sends the required authentication data. The image will default to run on port 8080, but you can customize it by setting the `PORT` environment variable when starting your image. ```bash docker run -e PORT=8081 -p 8081:8081 MY-IMAGE:TAG ``` Congratulations, you've built your own Docker image! 🎉 Continue to learn how to [configure infrastructure](https://encore.dev/docs/self-host/configure-infra). --- Source: https://encore.dev/docs/self-host/configure-infra Section: Deploying # Configure Infrastructure If you are using infrastructure resources, such as SQL databases, Pub/Sub, or metrics, you will need to configure your Docker image with the necessary configuration. The `build` command lets you provide this by specifying a path to a config file using the `--config` flag. ```bash encore build docker --config path/to/infra-config.json MY-IMAGE:TAG ``` The configuration file should be a JSON file using the [Encore Infra Config](https://encore.dev/schemas/infra.schema.json) schema. This supports configuring things like: - How to access infrastructure resources (what provider to use, what credentials to use, etc.) - How to call other services over the network ("service discovery"), most notably their base URLs. - Observability configuration (where to export metrics, etc.) - Metadata about the environment the application is running in, to power Encore's metadata APIs. - The values for any application-defined secrets. This configuration is necessary for the application to behave correctly. ## Example Here's an example configuration file you can use. ```json -- infra-config.json -- { "$schema": "https://encore.dev/schemas/infra.schema.json", "metadata": { "app_id": "my-app", "env_name": "my-env", "env_type": "production", "cloud": "gcp", "base_url": "https://my-app.com" }, "sql_servers": [ { "host": "my-db-host:5432", "databases": { "my-db": { "username": "my-db-owner", "password": {"$env": "DB_PASSWORD"} } } } ], "service_discovery": { "myservice": { "base_url": "https://myservice:8044" } }, "redis": { "encoreredis": { "database_index": 0, "auth": { "type": "acl", "username": "encoreredis", "password": {"$env": "REDIS_PASSWORD"} }, "host": "my-redis-host", } }, "metrics": { "type": "prometheus", "remote_write_url": "https://my-remote-write-url" }, "graceful_shutdown": { "total": 30 }, "auth": [ { "type": "key", "id": 1, "key": {"$env": "SVC_TO_SVC_KEY"} } ], "secrets": { "AppSecret": {"$env": "APP_SECRET"} }, "pubsub": [ { "type": "gcp_pubsub", "project_id": "my-project", "topics": { "encore-topic": { "name": "gcp-topic-name", "subscriptions": { "encore-subscription": { "name": "gcp-subscription-name" } } } } } ], "object_storage": [ { "type": "gcs", "buckets": { "my-gcs-bucket": { "name": "my-gcs-bucket", } } } ] } ``` ## Configuring Infrastructure To use infrastructure resources, additional configuration must be added so that Encore is aware of how to access each infrastructure resource. See below for examples of each type of infrastructure resource. ### 1. Basic Environment Metadata Configuration ```json { "metadata": { "app_id": "my-encore-app", "env_name": "production", "env_type": "production", "cloud": "aws", "base_url": "https://api.myencoreapp.com" } } ``` - `app_id`: The ID of your Encore application. - `env_name`: The environment name, such as `production`, `staging`, or `development`. - `env_type`: Specifies the type of environment (`production`, `test`, `development`, or `ephemeral`). - `cloud`: The cloud provider hosting the infrastructure (e.g., `aws`, `gcp`, or `azure`). - `base_url`: The base URL for services in the environment. ### 2. Graceful Shutdown Configuration ```json { "graceful_shutdown": { "total": 30, "shutdown_hooks": 10, "handlers": 20 } } ``` - `total`: The total time allowed for the shutdown process in seconds. - `shutdown_hooks`: The time allowed for executing shutdown hooks. - `handlers`: The time allocated for processing request handlers during the shutdown. ### 3. Authentication Methods Configuration Private endpoints will not require authentication if no authentication methods are specified. This is typically fine when services are deployed on a private network such as a VPC. But sometimes you might need to connect to other services over the public internet, in which case you'll want to ensure private endpoints are only accessible to other backend services. To do that you can configure authentication methods. Encore currently supports authentication through a shared key, which you can specify in your infrastructure configuration file. ```json { "auth": [ { "type": "key", "id": 1, "key": { "$env": "SERVICE_API_KEY" } } ] } ``` - `type`: The authentication method type (e.g., `key`). - `id`: The ID associated with the authentication method. - `key`: The authentication key, which can be set using an environment variable reference. ### 4. Service Discovery Configuration Service discovery is used to access other services over the network. You can configure service discovery in the infrastructure configuration file. If you export all services into the same docker image, you don't need to configure service discovery as it will be automatically configured when the services are started. ```json { "service_discovery": { "user-service": { "base_url": "https://user.myencoreapp.com", "auth": [ { "type": "key", "id": 1, "key": { "$env": "USER_SERVICE_API_KEY" } } ] } } } ``` - `user-service`: Configuration for a service named `user-service`. - `base_url`: The base URL for the service. - `auth`: Authentication methods used for accessing the service. If no authentication methods are specified, the service will use the auth methods defined in the `auth` section. ### 5. Metrics Configuration Similarly to cloud infrastructure resources, Encore supports configurable metrics exports: * Prometheus * DataDog * GCP Cloud Monitoring * AWS CloudWatch This is configured by setting the metrics field. Below are examples for each of the supported metrics providers: #### 5.1. Prometheus Configuration ```json { "metrics": { "type": "prometheus", "collection_interval": 15, "remote_write_url": { "$env": "PROMETHEUS_REMOTE_WRITE_URL" } } } ``` #### 5.2. Datadog Configuration ```json { "metrics": { "type": "datadog", "collection_interval": 30, "site": "datadoghq.com", "api_key": { "$env": "DATADOG_API_KEY" } } } ``` #### 5.3. GCP Cloud Monitoring Configuration ```json { "metrics": { "type": "gcp_cloud_monitoring", "collection_interval": 60, "project_id": "my-gcp-project", "monitored_resource_type": "gce_instance", "monitored_resource_labels": { "instance_id": "1234567890", "zone": "us-central1-a" }, "metric_names": { "cpu_usage": "compute.googleapis.com/instance/cpu/usage_time" } } } ``` #### 5.4. AWS CloudWatch Configuration ```json { "metrics": { "type": "aws_cloudwatch", "collection_interval": 60, "namespace": "MyAppMetrics" } } ``` ### 6. SQL Database Configuration The SQL databases you've declared in your Encore app must be configured in the infrastructure configuration file. There must be exactly one database configuration for each declared database. You can configure multiple SQL servers if needed. ```json { "sql_servers": [ { "host": "db.myencoreapp.com:5432", "tls_config": { "disabled": false, "ca": "---BEGIN CERTIFICATE---\n...", "disable_tls_hostname_verification": false, "disable_ca_validation": false }, "databases": { "main_db": { "name": "my-postgres-db-name", "max_connections": 100, "min_connections": 10, "username": "db_user", "password": { "$env": "DB_PASSWORD" } } } } ] } ``` - `host`: SQL server host, optionally including the port. - `tls_config`: TLS configuration for secure connections. If omitted, Encore tries TLS first and falls back to an unencrypted connection if the server doesn't support it. Setting `ca` makes TLS required. If the server uses TLS with a non-system CA root, or requires a client certificate, specify the appropriate fields as PEM-encoded strings. Otherwise, they can be left empty. The `disable_tls_hostname_verification` and `disable_ca_validation` options are only supported in Encore.ts. - `databases`: List of databases, each with connection settings. ### 7. Secrets Configuration #### 7.1. Using Direct Secrets You can set the secret value directly in the configuration file, or use an environment variable reference to set the secret value. ```json { "secrets": { "API_TOKEN": "embedded-secret-value", "DB_PASSWORD": { "$env": "DB_PASSWORD" } } } ``` #### 7.2. Using Environment Reference As an alternative, you can use an environment variable reference to set the secret value. The env variable should be set in the environment where the application is running. The content of the environment variable should be a JSON string where each key is the secret name and the value is the secret value. ```json { "secrets": { "$env": "SECRET_JSON" } } ``` ### 8. Redis Configuration **Encore.ts** ```json { "redis": { "cache": { "host": "redis.myencoreapp.com:6379", "database_index": 0, "auth": { "type": "auth_string", "auth_string": { "$env": "REDIS_AUTH_STRING" } }, "tls_config": { "disabled": false, "ca": "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----", "disable_tls_hostname_verification": false, "disable_ca_validation": false }, "max_connections": 50, "min_connections": 5 } } } ``` - `host`: Redis server host, optionally including the port. - `auth`: Authentication configuration for the Redis server. - `key_prefix`: Prefix applied to all keys. - `tls_config`: TLS configuration for the connection. If `tls_config` is omitted, TLS is required. Set `"disabled": true` if your Redis server doesn't use TLS. Set `ca` if the server uses a certificate signed by a non-system CA root. `disable_tls_hostname_verification` and `disable_ca_validation` turn off hostname and certificate checks. **Encore.go** ```json { "redis": { "cache": { "host": "redis.myencoreapp.com:6379", "database_index": 0, "auth": { "type": "auth_string", "auth_string": { "$env": "REDIS_AUTH_STRING" } }, "tls_config": { "disabled": false, "ca": "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----" }, "max_connections": 50, "min_connections": 5 } } } ``` - `host`: Redis server host, optionally including the port. - `auth`: Authentication configuration for the Redis server. - `key_prefix`: Prefix applied to all keys. - `tls_config`: TLS configuration for the connection. If `tls_config` is omitted, the connection doesn't use TLS. Set `"disabled": false` if your Redis server requires TLS. Set `ca` if the server uses a certificate signed by a non-system CA root. ### 9. Pub/Sub Configuration Encore currently supports the following Pub/Sub providers, set with `type`: - `nsq` for [NSQ](https://nsq.io/) - `gcp_pubsub` for [Google Cloud Pub/Sub](https://cloud.google.com/pubsub) - `aws_sns_sqs` for AWS [SNS](https://aws.amazon.com/sns/) + [SQS](https://aws.amazon.com/sqs/) The configuration for each provider is different. Below are examples for each provider. #### 9.1. GCP Pub/Sub ```json { "pubsub": [ { "type": "gcp_pubsub", "project_id": "my-gcp-project", "topics": { "user-events": { "name": "user-events-topic", "project_id": "my-gcp-project", "subscriptions": { "user-notification": { "name": "user-notification-subscription", "push_config": { "id": "user-push", "service_account": "service-account@my-gcp-project.iam.gserviceaccount.com", "jwt_audience": "https://api.myencoreapp.com" } } } } } } ] } ``` - `push_config`: Optional, for push subscriptions. If set, `id`, `service_account`, and `jwt_audience` are all required. `jwt_audience` must match the audience configured on the GCP push subscription. #### 9.2. AWS SNS/SQS ```json { "pubsub": [ { "type": "aws_sns_sqs", "topics": { "my-topic": { "arn": "arn:aws:sns:us-east-1:123456789012:my-topic", "subscriptions": { "my-queue": { "url": "https://sqs.eu-east-1.amazonaws.com/123456789012/my-queue" } } } } } ] } ``` - `my-topic`: This is the name of the topic as it is declared in your Encore app. - `my-queue`: This is the name of the queue as it is declared in your Encore app. - `arn`: The ARN of the SNS topic. - `url`: The URL of the SQS queue. #### 9.3. NSQ Configuration ```json { "pubsub": [ { "type": "nsq", "hosts": "nsq.myencoreapp.com:4150", "topics": { "order-events": { "name": "order-events-topic", "subscriptions": { "order-processor": { "name": "order-processor-subscription" } } } } } ] } ``` ### 10. Object Storage Configuration Encore currently supports the following object storage providers: - `gcs` for [Google Cloud Storage](https://cloud.google.com/storage) - `s3` for [AWS S3](https://aws.amazon.com/s3/) or a custom S3-compatible provider #### 10.1. GCS Configuration ```json { "object_storage": [ { "type": "gcs", "buckets": { "my-gcs-bucket": { "name": "my-gcs-bucket", "key_prefix": "my-optional-prefix/", "public_base_url": "https://my-gcs-bucket-cdn.example.com/my-optional-prefix" } } } ] } ``` - `name`: The full name of the GCS bucket. - `key_prefix`: An optional prefix to apply to all keys in the bucket. - `public_base_url`: A URL to use for public access to the bucket. This field is required if you configure your bucket to be public. Encore will append the object key to this URL when generating public URLs. The optional prefix will not be appended. #### 10.2. S3 Configuration ```json { "object_storage": [ { "type": "s3", "region": "us-east-1", "buckets": { "my-s3-bucket": { "name": "my-s3-bucket", "key_prefix": "my-optional-prefix/", "public_base_url": "https://my-gcs-bucket-cdn.example.com/my-optional-prefix" } } } ] } ``` - `region`: The AWS region where the bucket is located. - `name`: The full name of the S3 bucket. - `key_prefix`: An optional prefix to apply to all keys in the bucket. - `public_base_url`: A URL to use for public access to the bucket. This field is required if you configure your bucket to be public. Encore will append the object key to this URL when generating public URLs. The optional prefix will not be appended. #### 10.3. Custom S3 Provider Configuration You can also configure a custom S3 provider by specifying the endpoint, access key id, and secret access key. Custom S3 providers are useful if you are using a S3-compatible storage provider such as [Cloudflare R2](https://developers.cloudflare.com/r2/). ```json { "object_storage": [ { "type": "s3", "region": "auto", "endpoint": "https://...", "access_key_id": "...", "secret_access_key": { "$env": "BUCKET_SECRET_ACCESS_KEY" }, "buckets": { "my-s3-bucket": { "name": "my-s3-bucket", "key_prefix": "my-optional-prefix/", "public_base_url": "https://my-gcs-bucket-cdn.example.com/my-optional-prefix" } } } ] } ``` - `region`: The region where the bucket is located. - `name`: The full name of the bucket - `key_prefix`: An optional prefix to apply to all keys in the bucket. - `public_base_url`: A URL to use for public access to the bucket. This field is required if you configure your bucket to be public. Encore will append the object key to this URL when generating public URLs. The optional prefix will not be appended. This guide covers typical infrastructure configurations. Adjust according to your specific requirements to optimize your Encore app's infrastructure setup. --- Source: https://encore.dev/docs/self-host/ci-cd Section: Deploying # Integrate with your CI/CD pipeline Encore seamlessly integrates with any CI/CD pipeline through its CLI tools. You can automate Docker image creation using the `encore build` command as part of your deployment workflow. ## Integrating with CI/CD Platforms While every CI/CD pipeline is unique, integrating Encore follows a straightforward process. Here are the key steps: 1. Install the Encore CLI in your CI environment 2. Use `encore build docker` to create Docker images 3. Push the images to your container registry 4. Deploy to your infrastructure If your app is linked with the Encore platform, you'll need to authenticate the CLI in your CI environment using an [auth key](https://encore.dev/docs/platform/integrations/auth-keys). Generate one from **App Settings > Auth Keys** in the Encore platform dashboard, store it as a CI secret, and run `encore auth login --auth-key=` before building. Refer to your CI/CD platform's documentation for more details on how to integrate CLI tools like `encore build`. ### GitHub actions example This example shows how to build, push, and deploy an Encore Docker image to DigitalOcean using GitHub Actions. The DigitalOcean application is set up re-deploy the application every time an image with the tag `latest` is uploaded. ```yaml name: Build, Push and Deploy a Encore Docker Image to DigitalOcean on: push: branches: [ main ] permissions: contents: read packages: write jobs: build-push-deploy-image: runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v4 - name: Download Encore CLI script uses: sozo-design/curl@v1.0.2 with: args: --output install.sh -L https://encore.dev/install.sh - name: Install Encore CLI run: bash install.sh - name: Authenticate with Encore run: /home/runner/.encore/bin/encore auth login --auth-key=${{ secrets.ENCORE_AUTH_KEY }} - name: Log in to DigitalOcean container registry run: docker login registry.digitalocean.com -u my-email@gmail.com -p ${{ secrets.DIGITALOCEAN_ACCESS_TOKEN }} - name: Build Docker image run: /home/runner/.encore/bin/encore build docker --config infra.config.json myapp - name: Tag Docker image run: docker tag myapp registry.digitalocean.com//:latest - name: Push Docker image run: docker push registry.digitalocean.com//:latest ``` The `--config` flag points to your [infrastructure configuration](https://encore.dev/docs/self-host/configure-infra) file. For CI, commit only non-secret configuration and `$env` references; never put secret values in the committed file, and inject the referenced environment variables through your CI/deployment secret store. It's required if your app uses any infrastructure, such as databases or secrets. ## Building Docker Images The `encore build docker` command provides several options to customize your builds: ```bash # Build specific services and gateways encore build docker --services=service1,service2 --gateways=api-gateway MY-IMAGE:TAG # Customize the base image encore build docker --base=node:18-alpine MY-IMAGE:TAG # Build for a specific architecture (useful when CI and deploy targets differ) encore build docker --arch=arm64 MY-IMAGE:TAG ``` The image will default to run on port 8080, but you can customize it by setting the `PORT` environment variable when starting your image. ```bash docker run -e PORT=8081 -p 8081:8081 MY-IMAGE:TAG ``` Learn more about the `encore build docker` command in the [build Docker images](https://encore.dev/docs/self-host/build) guide. Continue to learn how to [configure infrastructure](https://encore.dev/docs/self-host/configure-infra). --- Source: https://encore.dev/docs/self-host/deploy-digitalocean Section: Deploying # Deploy to DigitalOcean If you prefer manual deployment over the automation offered by Encore's Platform, Encore simplifies the process of deploying your app to the cloud provider of your choice. This guide will walk you through deploying an Encore app to DigitalOcean's App Platform using Docker. ### Video tutorial ### Prerequisites 1. **DigitalOcean Account**: Make sure you have a DigitalOcean account. If not, you can [sign up here](https://www.digitalocean.com/). 2. **Docker Installed**: Ensure Docker is installed on your local machine. You can download it from the [Docker website](https://www.docker.com/get-started). 3. **Encore CLI**: Install the Encore CLI if you haven’t already. You can follow the installation instructions from the [Encore documentation](https://encore.dev/docs/quick-start). 4. **DigitalOcean CLI (Optional)**: You can install the DigitalOcean CLI for more flexibility and automation, but it’s not necessary for this tutorial. ### Step 1: Create an Encore App 1. **Create a New Encore App**: - If you haven’t already, create a new Encore app using the Encore CLI. - You can use the following command to create a new app: ```bash encore app create myapp ``` - Select the `Hello World` template. - Follow the prompts to create the app. 2. **Build a Docker image**: - Build the Encore app to generate the docker image for deployment: ```bash encore build docker myapp ``` ### Step 2: Push the Docker Image to a Container Registry To deploy your Docker image to DigitalOcean, you need to push it to a container registry. DigitalOcean supports its own container registry, but you can also use DockerHub or other registries. Here’s how to push the image to DigitalOcean’s registry: 1. **Create a DigitalOcean Container Registry**: - Go to the [DigitalOcean Control Panel](https://cloud.digitalocean.com/registries) and create a new container registry. - Follow the instructions to set it up. 2. **Login to DigitalOcean's registry**: Use the login command provided by DigitalOcean, which will look something like this: ```bash doctl registry login ``` You’ll need the DigitalOcean CLI for this, which can be installed from [DigitalOcean CLI documentation](https://docs.digitalocean.com/reference/doctl/how-to/install/). 3. **Tag your Docker image**: Tag your image to match the registry’s URL. ```bash docker tag myapp registry.digitalocean.com/YOUR_REGISTRY_NAME/myapp:latest ``` 4. **Push your Docker image to the registry**: ```bash docker push registry.digitalocean.com/YOUR_REGISTRY_NAME/myapp:latest ``` ### Step 3: Deploy the Docker Image to DigitalOcean App Platform 1. **Navigate to the App Platform**: Go to [DigitalOcean's App Platform](https://cloud.digitalocean.com/apps). 2. **Create a New App**: - Click on **"Create App"**. - Choose the **"DigitalOcean Container Registry"** option. 3. **Select the Docker Image Source**: - Select the image you pushed earlier. 4. **Configure the App Settings**: - **Set up scaling options**: Configure the number of containers, CPU, and memory settings. - **Environment variables**: Add any environment variables your application might need. - **Choose the region**: Pick a region close to your users for better performance. 5. **Deploy the App**: - Click **"Next"**, review the settings, and click **"Create Resources"**. - DigitalOcean will take care of provisioning the infrastructure, pulling the Docker image, and starting the application. ### Step 4: Monitor and Manage the App 1. **Access the Application**: - Once deployed, you will get a public URL to access your application. - Test the app to ensure it’s running as expected, e.g. ```bash curl https://myapp.ondigitalocean.app/hello/world ``` 2. **View Logs and Metrics**: - Go to the **"Runtime Logs"** tab in the App Platform to view logs - Go to the **"Insights"** tab to view performance metrics. 3. **Manage Scaling and Deployment Settings**: - You can change the app configuration, such as scaling settings, deployment region, or environment variables. ### Step 5: Add a Database to Your App DigitalOcean’s App Platform provides managed databases, allowing you to add a database to your app easily. Here’s how to set up a managed database for your app: 1. **Navigate to the DigitalOcean Control Panel**: - Go to [DigitalOcean Control Panel](https://cloud.digitalocean.com/). - Click on **"Databases"** in the left-hand sidebar. 2. **Create a New Database Cluster**: - Click **"Create Database Cluster"**. - Choose **PostgreSQL** - Select the **database version**, **data center region**, and **cluster configuration** (e.g., development or production settings based on your needs). - **Name the database** and configure other settings if necessary, then click **"Create Database Cluster"**. 3. **Configure the Database Settings**: - Once the database is created, go to the **"Connection Details"** tab of the database dashboard. - Copy the **connection string** or individual settings (host, port, username, password, database name). You will need these details to connect your app to the database. - Download the **CA certificate** 4. **Create a Database** - Connect to the database using the connection string provided by DigitalOcean. ```bash psql -h mydb.db.ondigitalocean.com -U doadmin -d mydb -p 25060 ``` - Create a database ```sql CREATE DATABASE mydb; ``` - Create a table ```sql CREATE TABLE users ( id SERIAL PRIMARY KEY, name VARCHAR(50) ); INSERT INTO users (name) VALUES ('Alice'); ``` 5. **Declare a Database in your Encore app**: - Open your Encore app’s codebase. - Add `mydb` database to your app ([Encore Database Documentation](https://encore.dev/docs/ts/primitives/databases)) ```typescript const mydb = new SQLDatabase("mydb", { migrations: "./migrations", }); export const getUser = api( { expose: true, method: "GET", path: "/names/:id" }, async ({id}: {id:number}): Promise<{ id: number; name: string }> => { return await mydb.queryRow`SELECT * FROM users WHERE id = ${id}` as { id: number; name: string }; } ); ``` 6. **Create an Encore Infrastructure config** - Create a file named `infra.config.json` in the root of your Encore app. - Add the **CA certificate** and the connection details to the file: ```json { "$schema": "https://encore.dev/schemas/infra.schema.json", "sql_servers": [ { "host": "mydb.db.ondigitalocean.com:25060", "tls_config": { "ca": "-----BEGIN CERTIFICATE-----\n..." }, "databases": { "mydb": { "name": "mydb", "username": "doadmin", "password": {"$env": "DB_PASSWORD"} } } }] } ``` 7. **Set Up Environment Variables (Optional)**: - Go to the DigitalOcean App Platform dashboard. - Select your app. - In the **"Settings"** section, go to **"App-Level Environment Variables"** - Add the database password as an encrypted environment variable called `DB_PASSWORD`. 8. **Build and push the Docker image**: - Build the Docker image with the updated configuration. ```bash encore build docker --config infra.config.json myapp ``` - Tag and push the Docker image to the DigitalOcean container registry. ```bash docker tag myapp registry.digitalocean.com/YOUR_REGISTRY_NAME/myapp:latest docker push registry.digitalocean.com/YOUR_REGISTRY_NAME/myapp:latest ``` 9. **Test the Database Connection**: - Redeploy the app on DigitalOcean to apply the changes. - Test the database connection by calling the API ```bash curl https://myapp.ondigitalocean.app/names/1 ``` ### Troubleshooting Tips - **Deployment Failures**: Check the build logs for any errors. Make sure the Docker image is correctly tagged and pushed to the registry. - **App Not Accessible**: Verify that the correct port is exposed in the Dockerfile and the App Platform configuration. - **Database Connection Issues**: Ensure the database connection details are correct and the database is accessible from the app. ### Conclusion That’s it! You’ve successfully deployed an Encore app to DigitalOcean’s App Platform using Docker. You can now scale your app, monitor its performance, and manage it easily through the DigitalOcean dashboard. If you encounter any issues, refer to the DigitalOcean documentation or the Encore community for help. Happy coding! --- Source: https://encore.dev/docs/self-host/deploy-railway Section: Deploying # Deploy to Railway If you prefer manual deployment over the automation offered by Encore's Platform, Encore simplifies the process of deploying your app to the cloud provider of your choice. This guide will walk you through deploying an Encore app to Railway using Docker through GitHub Actions. ### Prerequisites 1. **Railway Account**: Make sure you have a Railway account. If not, you can [sign up here](https://railway.com/). 2. **Docker Installed**: Ensure Docker is installed on your local machine, Docker is used by Encore to run databases locally. You can download it from the [Docker website](https://www.docker.com/get-started). 3. **Encore CLI**: Install the Encore CLI if you haven’t already. You can follow the installation instructions from the [Encore documentation](https://encore.dev/docs/quick-start). ### Step 1: Create an Encore App and a GitHub repository 1. **Create a New Encore App**: - Create a new Encore app using the Encore CLI by running the following command: ```bash encore app create ``` - Select the `Hello World` template. - Follow the prompts to create the app. 2. **Push the code to a GitHub repo**: - Create a new repo (public or private) on GitHub and push the code to it. ### Step 2: Push the Docker Image to GitHub's Container Registry To deploy your Docker image to Railway, you first need to push it to a container registry. We will be using GitHub's container registry, but you can also use DockerHub or other registries. Instead of pushing the image manually we will be using GitHub actions to automate the process. 1. **Create a GitHub Actions YAML file**: - In your repo, create a `.github/workflows/deploy-image-yaml` file with the following contents: ```yaml name: Build, Push and Deploy a Docker Image to Railway on: push: branches: [ main ] permissions: contents: read packages: write jobs: build-push-deploy-image: runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v4 - name: Log in to the Container registry uses: docker/login-action@v3.3.0 with: registry: ghcr.io username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - name: Download Encore CLI script uses: sozo-design/curl@v1.0.2 with: args: --output install.sh -L https://encore.dev/install.sh - name: Install Encore CLI run: bash install.sh - name: Build Docker image run: /home/runner/.encore/bin/encore build docker myapp - name: Tag Docker image run: docker tag myapp ghcr.io/${{ github.repository }}:latest - name: Push Docker image run: docker push ghcr.io/${{ github.repository }}:latest ``` This will install the Encore CLI, build the Docker image, tag it, and push it to GitHub's container registry everytime you push to the `main` branch. The dynamic values like `${{ github.repository }}` will be filled in automatically by GitHub, you should not need to do anything. 2. **Add, commit and push the changes**: - Push the changes to your GitHub repository to trigger the GitHub action. ### Step 3: Deploy the Docker Image to Railway 1. **Create a new Project on Railway**: - Log in to Railway and go to your dashboard. - Click on **"New"**. - Choose the **"Empty project"** option. 2. **Create a new service inside your new project**: - Click on **"Create"**. - Select the "Docker Image" option. - Enter the Docker Image URI, should be something like `ghcr.io/username/repo:latest`. You can should be able to find the Docker Image under **Packages** in your GitHub repo. - Deploy the service. 3. **Expose the service**: - Click on the tne newly created service. - Go to the **"Settings"** tab. - Click on **"Generate Domain"**. - Select `8080` as the port. - Click on **"Generate"**. 4. **Access the application**: - Once deployed, and exposed you will get a public URL to access your application. It should look something like this: `https://repo-name-production.up.railway.app/`. - Test the app to ensure it's running as expected, e.g. ```bash curl https://repo-name-production.up.railway.app/hello/world ``` ### Step 4: Automate the Deployment Process Railway has no way of knowing that you've pushed a new image to the container registry, but we can use Railway's GraphQL API to trigger a new deployment whenever a new image is pushed to the registry. 1. **Generate a Railway API Token**: - Go to your Railway dashboard. - Click on your profile icon in the top right corner. - Go to **"Account Settings"**. - Click on **"Tokens"**. - Give the token a name and click on **"Create"**. - Copy the generated token. 2. **Add the Railway API Token to GitHub Secrets**: - Go to your GitHub repository. - Go to **"Settings"**. - Click on **"Secrets and variables" → "Actions"**. - Click on **"New repository secret"**. - Add a new secret called `RAILWAY_API_TOKEN` and paste the token you copied earlier. 3. **Add a JavaScript script to your repo**: - Create a new file in your repo named `script.js` with the following contents: ```javascript -- script.js -- const TOKEN = process.argv.slice(2)[0]; const ENVIRONMENT_ID = "" const SERVICE_ID = "" const resp = await fetch('https://backboard.railway.com/graphql/v2', { method: 'POST', headers: { 'Content-Type': 'application/json', 'authorization': `Bearer ${TOKEN}`, }, body: JSON.stringify({ query: ` mutation ServiceInstanceRedeploy { serviceInstanceRedeploy( environmentId: "${ENVIRONMENT_ID}" serviceId: "${SERVICE_ID}" ) }` }), }) const data = await resp.json() if (data.errors) { console.error(data.errors) throw new Error('Failed to redeploy service') } console.log(data) ``` - Replace `` and `` with the actual values. You can find these values in the Railway dashboard URL when you're on the service page. 4. **Add new steps to the GitHub Actions YAML file**: - At the bottom of the existing file, add the following steps to call the script: ```yaml - name: Set up Node uses: actions/setup-node@v4 with: node-version: 22 - name: Trigger Railway deployment run: node script.js ${{ secrets.RAILWAY_API_TOKEN }} ``` Whenever you push a new Docker Image to the container registry, the GitHub action will trigger a new deployment on Railway. ### Step 5: Add a Database to Your App Railway provides managed databases, allowing you to add a database to your app easily. Here’s how to set up a database for your app: 1. **Create a database for your app on Railway**: - Navigate to your Railway app. - Click on **"Create"** → **"Database""** → **"Add PostgreSQL""** 2. **Copy the connection details**: - Click on the database you just created. - Click the **"Data"** → **"Connect"** → **"Public Network"**. - Copy the raw `psql` command connection details. 3. **Create a database table**: - Connect to the database using the `psql` command: ```bash PGPASSWORD= psql -h .rlwy.net -U postgres -p 39684 -d railway ``` - Create a table ```sql CREATE TABLE users ( id SERIAL PRIMARY KEY, name TEXT ); INSERT INTO users (name) VALUES ('Alice'); ``` 4. **Declare a Database in your Encore app**: - Open your Encore app’s codebase. - Add `mydb` database to your app ([Encore Database Documentation](https://encore.dev/docs/ts/primitives/databases)) ```typescript const mydb = new SQLDatabase("mydb", { migrations: "./migrations", }); export const getUser = api( { expose: true, method: "GET", path: "/names/:id" }, async ({id}: {id:number}): Promise<{ id: number; name: string }> => { return await mydb.queryRow`SELECT * FROM users WHERE id = ${id}` as { id: number; name: string }; } ); ``` 5. **Create an Encore Infrastructure config** - Create a file named `infra.config.json` in the root of your Encore app. - Add the connection details to the file: ```json { "$schema": "https://encore.dev/schemas/infra.schema.json", "sql_servers": [ { "host": ".rlwy.net:39684", "tls_config": { "disable_ca_validation": true }, "databases": { "mydb": { "name": "railway", "username": "postgres", "password": {"$env": "DB_PASSWORD"} } } }] } ``` Railway does not allow for downloading the CA certificate for the database, so we disable the CA validation. 6. **Pass the config to the build**: - In `.github/workflows/deploy-image-yaml`, update the **Build Docker image** step to use the config file: ```yaml - name: Build Docker image run: /home/runner/.encore/bin/encore build docker --config infra.config.json myapp ``` 7. **Set Up Environment Variables**: - Click on the deployed image in your app view on Railway. - Click **"Variables"**. - Add the database password as an environment variable called `DB_PASSWORD`. 8. **Make a new deployment**: - Add commit and push the changes to your GitHub repository, this will trigger a new deploy on Railway. 9. **Test the Database Connection**: - Test the database connection by calling the API ```bash curl https://myapp.railway.app/names/1 ``` ### Conclusion That’s it! You’ve successfully deployed an Encore app to Railway using Docker. You can now scale your app, monitor its performance, and manage it easily through the Railway dashboard. If you encounter any issues, refer to the Railway documentation or the Encore community for help. Happy coding! --- Source: https://encore.dev/docs/service-catalog Section: Observability # Service Catalog > Automatically get a Service Catalog and complete API docs All developers agree API documentation is great to have, but the effort of maintaining it inevitably leads to docs becoming stale and out of date. To solve this, Encore uses the [Encore Application Model](https://encore.dev/docs/understanding-encore) to automatically generate a Service Catalog along with complete documentation for all APIs. This ensures docs are always up-to-date as your APIs evolve. The API docs are available both in your [Local Development Dashboard](https://encore.dev/docs/dev-dash) and for your whole team in the [Encore platform dashboard](https://app.encore.dev). To learn how to write doc comments that are surfaced in the Service Catalog, generated clients, and OpenAPI specs, see the API Documentation guide for [TypeScript](https://encore.dev/docs/ts/develop/api-docs) or [Go](https://encore.dev/docs/go/develop/api-docs). --- Source: https://encore.dev/docs/tracing Section: Observability # Distributed Tracing > Track requests across your application and infrastructure Distributed systems often have many moving parts, making it difficult to understand what your code is doing and finding the root-cause to bugs. That’s where Tracing comes in. If you haven’t seen it before, it may just about change your life. Tracing is a revolutionary way to gain insight into what your applications are doing. It works by capturing the series of events as they occur during the execution of your code (a “trace”). This works by propagating a trace id between all individual systems, then correlating and joining the information together to present a unified picture of what happened end-to-end. As opposed to the labor intensive instrumentation you'd normally need to go through to use tracing, Encore automatically captures traces for your entire application – in all environments. Uniquely, this means you can use tracing even for local development to help debugging and speed up iterations. You view traces in the [Local Development Dashboard](https://encore.dev/docs/dev-dash) and in the [Encore platform dashboard](https://app.encore.dev) for Production and other environments. ## Encore's tracing is more comprehensive and more performant than all other tools Unlike other tracing solutions, Encore understands what each trace event is and captures unique insights about each one. This means you get access to more information than ever before: * Stack traces * Structured logging * HTTP requests * Network connection information * API calls * Database queries * etc. ## Redacting sensitive data Encore's tracing automatically captures request and response payloads to simplify debugging. For cases where this is undesirable, such as for passwords or personally identifiable information (PII), Encore supports redacting fields marked as containing sensitive data. See the documentation on [API Schemas](https://encore.dev/docs/ts/primitives/defining-apis#sensitive-data) for more information. --- Source: https://encore.dev/docs/architecture Section: Observability # Architecture Diagram > Visualize your cloud microservices architecture The architecture diagram is a visual tool that gives you an always up-to-date view of your entire system, helping you reason about your microservices architecture and identify which services depend on each other and how they work together. ## Birds-eye view Having access to a zoomed out representation of your system can be invaluable in pretty much all parts of the development cycle. The architecture diagram helps you: * Track down bottlenecks before they grow into big problems. * Get new team members onboarded much faster. * Pinpoint hot paths in your system, services that might need extra attention. Services and PubSub topics are represented as boxes, arrows indicate a dependency. In the example below the `login` service has dependencies on the `user` and `authentication` services. Dashed arrows shows publications or subscriptions to a topic. Here, `payment` publishes to the `payment-made` topic and `email` subscribe to it: ![Architecture Diagram - Highlight Dependencies](https://encore.dev/assets/docs/flow-diagram.png) ## Highlight dependencies Hover over a service, or PubSub topic, to instantly reveal the nature and scale of its dependencies. Here the `login` service and its dependencies are highlighted. We can see that `login` makes queries to the database and requests to two of the endpoints from the `user` service as well as requests to one endpoint from the `authentication` service: ![Architecture Diagram - Highlight Dependencies](https://encore.dev/assets/docs/flow-highlight.png) ## Real-time updates The architecture diagram is accessible in the [Local Development Dashboard](https://encore.dev/docs/dev-dash) and the [Encore platform dashboard](https://app.encore.dev) for cloud environments. When developing locally, the architecture diagram will auto update in real-time to reflect your architecture as you make code changes. This helps you be mindful of important dependencies and makes it clear if you introduce new ones. For cloud environments, the architecture diagram auto-updates with each deploy. In the example below a new subscription on the topic `payment-made` is introduced and then removed in `user` service: --- Source: https://encore.dev/docs/platform/deploy/security Section: Operations # Application Security > The Encore platform makes strong security the default path ## Built on industry experience The security practices in the Encore platform are built on our team's decades of experience designing and operating sensitive systems at companies like Google, Spotify, and Monzo. ## Security by Default The Encore platform is designed to make security effortless rather than burdensome: - **Zero-config security**: Focus on building features while the Encore platform automatically implements security best practices - **Built-in secrets management**: Safely handle sensitive data using the built-in [secrets management system](https://encore.dev/docs/ts/primitives/secrets) - **Automated IAM management**: the Encore platform automatically manages IAM policies based on the principle of least privilege ## Security features When the Encore platform deploys your application and infrastructure, it takes care of implementing security best practices: - **Secure databases**: Database access is encrypted with certificate validation and strong security credentials - **Isolated database credentials**: Each database instance has unique credentials, and each container connecting to a database uses its own credential. Credentials can be [rotated via the dashboard](https://encore.dev/docs/platform/infrastructure/manage-db-users). - **Cloud security**: Automatic provisioning with security best practices specific to each cloud provider - Learn more about [Google Cloud Platform (GCP)](https://encore.dev/docs/platform/infrastructure/gcp) - Learn more about [Amazon Web Services (AWS)](https://encore.dev/docs/platform/infrastructure/aws) - **Infrastructure safety**: Deletion protection, admin-only environment management, and full audit trails for infrastructure changes. Learn more in [Managing Infrastructure](https://encore.dev/docs/platform/infrastructure/managing-infrastructure). --- Source: https://encore.dev/docs/platform/observability/tracing Section: Operations # Trace Sampling & Budgets > Control tracing volume and spend in cloud environments The Encore platform builds on the tracing described in [Distributed Tracing](https://encore.dev/docs/tracing), adding controls for how much of it is recorded and what it costs. ## Trace Sampling Trace sampling lets you control what percentage of traces are recorded and stored. You can configure sampling rates per environment, service, and endpoint, giving you fine-grained control over your tracing volume. ### How sampling works Sampling is determined at the root of the trace. This means if you set an endpoint to sample at 10%, it controls whether a trace is created when that endpoint is called as the initial entry point. If that same endpoint is called as part of an already-ongoing trace (e.g. as an internal service-to-service call), it will always be included in the existing trace regardless of its own sampling rate. This design ensures that all traces are complete — you'll never see partial traces with missing spans. Either a trace is sampled in its entirety, or not at all. ### Configuring sampling rates You can configure sampling rates in the Encore platform dashboard. Sampling can be set at three levels of granularity: - **Environment level**: Set a default sampling rate for all traces in an environment. - **Service level**: Override the environment default for a specific service. - **Endpoint level**: Override the service default for a specific endpoint. More specific settings take precedence. For example, if your environment is set to sample 100% of traces but a high-traffic endpoint is set to 10%, that endpoint will only generate new traces 10% of the time it's called as the root of a request. ## Trace Budgets Trace budgets give you full predictability over your tracing costs by letting you set spending limits on a daily and monthly basis. When a budget limit is reached, tracing is paused until the next period begins, ensuring you never receive unexpected charges. ### Included events The Encore platform includes a generous amount of tracing events in each plan: - **Free tier**: 1M trace events per month included. - **Pro tier**: 20M trace events per month included. Beyond the included events, Pro tier usage is billed at **$1.20 per million events**. ### Setting budgets You can configure your trace budgets in the Encore platform dashboard. By setting daily and monthly limits, you define exactly how much you're willing to spend on tracing. This makes tracing costs fully predictable and prevents any surprises on your bill. --- Source: https://encore.dev/docs/platform/observability/metrics Section: Operations # Metrics Integrations > Export your metrics to third-party observability services The Encore platform can forward the metrics your application exposes to third-party observability services. For how to define metrics in your code, see the SDK's metrics documentation. ## Integrations with third party observability services To make it easy to use a third party service for monitoring, we're adding direct integrations between Encore and popular observability services. This means you can send your metrics directly to these third party services instead of your cloud provider's monitoring service. ### Grafana Cloud To send metrics data to Grafana Cloud, you first need to Add a Grafana Cloud Stack to your application. Open your application in the [Encore platform dashboard](https://app.encore.dev), and click on **Settings** in the main navigation. Then select **Grafana Cloud** in the settings menu and click on **Add Stack**. ![Add a Grafana Stack](https://encore.dev/assets/docs/grafanastack.png) Next, open the environment **Overview** for the environment you wish to sent metrics from and click on **Settings**. Then in the **Sending metrics data** section, select your Grafana Cloud Stack from the drop-down and save. ![Select Grafana Stack](https://encore.dev/assets/docs/configstack.png) That's it! After your next deploy, Encore will start sending metrics data to your Grafana Cloud Stack. > **Note** > > To configure Encore to export metrics to Grafana Cloud, create a token with the following steps: > > 1. In Grafana, navigate to **Administration > Users and access > Cloud access policies** > 2. Click **Create access policy**, select **metrics:read** and **metrics:write** scopes, then click **Create** > 3. On the newly created access policy, click **Add token**, then **Create** to generate the token ### Datadog To send metrics data to Datadog, you first need to add a Datadog Account to your application. Open your application in the [Encore platform dashboard](https://app.encore.dev), and click on **Settings** in the main navigation. Then select **Datadog** in the settings menu and click on **Add Account**. ![Add a Datadog account](https://encore.dev/assets/docs/datadogaccount.png) Next, open the environment **Overview** for the environment you wish to sent metrics from and click on **Settings**. Then in the **Sending metrics data** section, select your Datadog Account from the drop-down and save. ![Select Datadog Account](https://encore.dev/assets/docs/configstack.png) That's it! After your next deploy, Encore will start sending metrics data to your Datadog Account. --- Source: https://encore.dev/docs/platform/infrastructure/managing-infrastructure Section: Operations # Managing Infrastructure > Day 2 operations for production infrastructure The Encore platform provides built-in safeguards to protect your production infrastructure, manages upgrades across the stack, and gives you the tools to implement disaster recovery. This page covers the operational aspects of running production infrastructure with the Encore platform. ## Infrastructure Safety ### Deletion protection The Encore platform implements multiple layers of protection against accidental or malicious deletion of production resources: - **Admin-only environment deletion:** Only users with the Admin role can destroy or delete environments. [Learn more about roles](https://encore.dev/docs/platform/management/permissions). - **Confirmation required:** Destroying an environment requires the admin to manually type a confirmation message. - **Stateful resource protection:** Stateful resources (databases, buckets, queues) are never deleted unless they are unused by the application and a user has manually confirmed deletion. Encore tracks which resources are in use through its [Application Model](https://encore.dev/docs/understanding-encore). - **Cloud provider safeguards:** For deployments to your own cloud, SQL databases have deletion protection enabled at the cloud provider level, which must be manually disabled in the cloud provider console before the resource can be removed. ### Drift detection The Encore platform is designed to coexist with manual changes made in your cloud provider's console. When deploying, Encore pulls the current resource properties before making any changes. If drift is detected (i.e. the resource was modified outside of Encore), Encore updates its internal representation to match the current state, unless there is a pending, manually requested property change in the Encore platform dashboard that hasn't been applied yet. This means you can safely make changes directly in your cloud provider's console without worrying about Encore overwriting them. Learn more in the [Infrastructure Configuration](https://encore.dev/docs/platform/infrastructure/configuration) docs. ### Audit trail The Encore platform maintains a versioned infrastructure graph where each node and edge is an immutable, versioned entity. Changes are applied by adding a new version, creating a complete audit trail: - Each infrastructure graph version is linked to the deployment that applied it. - Each deployment is connected to a specific git commit and any manually requested infrastructure changes. - Each infrastructure change request is linked to the user who requested it. - If a deployment was triggered manually (rather than via git push), the triggering user is also recorded. This gives you full traceability from any infrastructure state back to the code change, change request, and user responsible. ## Deployment Phases When the Encore platform deploys your application, it follows a three-phase process: 1. **Infrastructure provisioning:** Encore sets up any new infrastructure required by your code changes, such as databases, IAM policies, Pub/Sub topics, and other resources. 2. **Deployment:** Encore deploys the new container images to your compute platform (Cloud Run, Fargate, Kubernetes, etc.). 3. **Cleanup:** Encore removes any unused stateless resources. Stateful resources are only removed if they are unused and deletion has been explicitly confirmed by a user. ### Deploy approval For production environments, you can require admin approval before any deployment that includes infrastructure changes. When enabled, an Admin must manually review and approve the changes before the deployment proceeds. This includes IAM changes. To configure this, go to **Encore Cloud dashboard > (Select your environment) > Settings > Infrastructure Approval**. Learn more in the [Environments](https://encore.dev/docs/platform/deploy/environments) docs. ## Control Plane Separation The Encore platform separates the control plane from the runtime of your services. Your application runs independently in your own cloud environment, so if the Encore platform's control plane is unavailable, your production services continue to operate normally. The main impact of a control plane outage would be on control plane features like deployments, observability, and the Encore platform dashboard. Your running services, databases, and other infrastructure remain unaffected. For recovery independently of Encore's service, you retain full ownership of your infrastructure and code. You can use the [open source tools](https://encore.dev/docs/self-host/build) to build images and deploy using any alternative CI/CD tooling. ## Upgrades ### Runtime and SDK upgrades Encore runtime versions are automatically included as part of each build in the CI/CD pipeline. When you deploy, Encore builds your application with the latest runtime. - **Backward compatibility:** The Encore SDK maintains backward compatibility. Breaking changes have never been introduced, and if one were necessary, it would be communicated ahead of time through release notes and directly to all customers on paid plans. - **Custom runtime versions:** You can customize the Docker base image used for deployments through the `build.docker.base_image` setting in your `encore.app` file. This lets you pin a specific runtime version if needed. Learn more in the [Deploying](https://encore.dev/docs/platform/deploy/deploying) docs. - **Rollback:** To roll back a runtime version, update the configuration to specify the desired version and re-deploy. - **Testing upgrades:** While generally no action is required due to backward compatibility, you can verify behavior by deploying to a staging/development environment first. ### Database upgrades (PostgreSQL) The Encore platform automatically applies schema migrations as part of each deployment. However, PostgreSQL major version upgrades (e.g. PostgreSQL 15 to 16) are handled differently: - **Major version upgrades** must be manually initiated. They involve downtime and should be planned accordingly. Expect 10 to 20 minutes of downtime depending on database size, controlled by the cloud provider. - **Migration tooling:** Standard cloud provider tools and PostgreSQL utilities like `pg_dump`/`pg_restore` are supported for migrating data between database versions or environments. - **Testing:** Always test major upgrades in a staging environment before rolling out to production. Major PostgreSQL version upgrades may introduce backward compatibility issues at the SQL level. ### Infrastructure component upgrades Upgrades to the underlying cloud services (Cloud Run, Pub/Sub, IAM, VPC, etc.) are handled as part of the normal [deployment phases](#deployment-phases). By default, these upgrades are applied automatically during deployment. You can require admin approval for deployments that include infrastructure changes through the [deploy approval](#deploy-approval) settings. ## Database Management ### Instance naming The Encore platform assigns names to database instances automatically when provisioning. Cloud providers (e.g. GCP Cloud SQL, AWS RDS) do not support renaming instances after creation. If you need a different naming convention (for example, to include environment or service identifiers), the process involves: 1. Cloning the database to a new instance with the desired name. 2. Contacting Encore support to reconfigure the environment to point to the new instance. 3. Planning for downtime while the instance is being replaced. ### Database schemas Encore does not enforce a specific PostgreSQL schema for your tables. Table creation is controlled entirely by your [migration files](https://encore.dev/docs/ts/primitives/databases#database-migrations), so you have full control over which schema tables are created in. If your organization requires tables to be in a dedicated schema rather than `public`, update your migration files accordingly. For example: ```sql CREATE SCHEMA IF NOT EXISTS myapp; CREATE TABLE myapp.todo_item ( id BIGSERIAL PRIMARY KEY, title TEXT NOT NULL ); ``` This is purely a migration-level decision and does not require any changes to Encore's configuration. ### Database availability The Encore platform provisions databases with regional (multi-AZ) availability by default, distributing replicas across multiple availability zones for higher resilience. This means your database remains available even if an individual zone experiences an outage. You can change the availability configuration on the **Infrastructure** page in the Encore platform dashboard: - **Regional (multi-AZ):** Default. Provides higher availability and automatic failover across zones. Recommended for production workloads. - **Zonal (single-AZ):** Runs the database in a single availability zone. Lower cost, but no automatic failover if the zone goes down. Suitable for development environments or workloads where cost is a higher priority than availability. > **Important** > > Switching between regional and zonal availability causes downtime while the change is applied. Plan accordingly and consider making this change during a maintenance window. ### Credential management The Encore platform automatically manages database credentials with built-in isolation: - **Per-instance credentials:** Each database instance has its own unique credentials. - **Per-container credentials:** Each container connecting to a database instance uses a unique credential. - **Credential rotation:** Credentials can be rotated through the Encore platform dashboard. Navigate to the **Infrastructure** page for the relevant environment and use the rotation controls in the database cluster section. Learn more about viewing and managing database credentials in the [Managing database users](https://encore.dev/docs/platform/infrastructure/manage-db-users) docs. ### Connecting external tools to databases When deploying to your own cloud account, the Encore platform provisions databases in private subnets that are not directly accessible from the public internet. If you need to connect external tools (data pipelines, BI platforms, database management utilities, etc.) to your databases, there are a few approaches: **Using the Encore CLI proxy** The simplest way to connect external tools is through the Encore CLI's built-in database proxy: ```shell $ encore db proxy --env= ``` This sets up a local proxy that forwards connections to your databases. External tools can then connect to the local proxy endpoint. Learn more in the [database CLI docs](https://encore.dev/docs/ts/primitives/databases#connecting-to-databases). **Using database credentials directly** For cloud environments on AWS/GCP, you can retrieve database credentials from the Encore platform dashboard and use them with your cloud provider's connectivity options (e.g. VPC peering, private service access, or a proxy). See the [Managing database users](https://encore.dev/docs/platform/infrastructure/manage-db-users) docs for how to access credentials. **SSL/TLS certificates** In GCP, the Encore platform provisions databases with TLS encryption enabled. When connecting external tools (data pipelines, replication services, database management utilities, etc.), you will need to configure SSL certificates for the connection. To download the CA certificate and create a client certificate for your database instance: *GCP (Cloud SQL):* 1. Go to the Cloud SQL Instances page in the GCP console. 2. Click on your PostgreSQL instance name. 3. Select **Connections** from the left-hand menu. 4. Click the **Security** tab. 5. Under **Manage certificates**, click **Download CA certificate**. This downloads the `server-ca.pem` file. 6. If your tool requires a client certificate, under **Manage client certificates**, click **Create Client Certificate** and follow the prompts. Configure your external tool to trust the downloaded CA certificate, and provide the client certificate if required. The exact configuration depends on the tool you are using. If your tool connects over a trusted internal network (e.g. within the same VPC via a private IP or a secured proxy), you may be able to disable TLS verification. This is only appropriate when the connection path is already encrypted or isolated at the network level. Tools connecting over the public internet should always use TLS with proper certificate validation. ## Disaster Recovery ### Built-in protections When deploying to your own cloud account, the Encore platform provisions databases with the following protections by default: - **Automated daily backups** retained for 7 days - **Point-in-time recovery (PITR)** capabilities - **Private subnet placement** for network isolation Learn more about the default database configuration in the [GCP Infrastructure](https://encore.dev/docs/platform/infrastructure/gcp) and [AWS Infrastructure](https://encore.dev/docs/platform/infrastructure/aws) docs. ### Configuring DR Disaster recovery settings for stateful resources can be configured in two ways: 1. **Encore platform dashboard:** Use the infrastructure configuration UI to adjust backup schedules, retention periods, and other DR-related settings. 2. **Cloud provider console:** Since the Encore platform deploys infrastructure directly into your cloud account, you can configure DR settings (high availability, multi-zone, cross-region replication, etc.) directly in your cloud provider's console. Learn more about manual configuration in the [Infrastructure Configuration](https://encore.dev/docs/platform/infrastructure/configuration) docs. ### Restoring from backup The restore flow differs between cloud providers, since each handles backup restoration differently. In both cases, restores are initiated from the cloud provider's console using the automated backups or PITR snapshots that the Encore platform provisions by default. **GCP (Cloud SQL)** Cloud SQL restores in place: the backup is applied to the existing instance, overwriting its current contents. No changes are needed on the Encore platform side. Once the restore completes in the GCP console, your application will be using the restored data automatically, since the instance identifier and connection details remain unchanged. **AWS (RDS)** RDS restores create a new instance (or cluster) rather than overwriting the existing one. After the restore completes in the AWS console, you need to point the Encore platform at the new cluster so that your application connects to it: 1. Go to the **Encore Cloud dashboard > (Select your environment) > Infrastructure**. 2. Locate the RDS instance and click **Edit**. 3. Click **Failover cluster** and enter the instance ID of the restored cluster. 4. Save the change and deploy to apply. After the deployment, your application will connect to the restored cluster. Keep the original cluster running until you have verified that the restored data is correct and the application is healthy. ### Recommended practices For production environments with specific RTO/RPO targets, consider: - Configuring automated backups with appropriate retention periods - Enabling point-in-time recovery for databases - Setting up high-availability and multi-zone configurations - Testing recovery procedures regularly in non-production environments - Documenting your recovery runbooks ## What the Encore platform Automates vs. What You Control The Encore platform automates the heavy lifting of infrastructure management, while giving you full control over customization and decisions that are specific to your application. ### Automated by the Encore platform - **Infrastructure provisioning:** Cloud resources are automatically created and managed based on your application code. No Terraform, no YAML. - **Deployments and CI/CD:** Every git push triggers a build, test, and deploy pipeline. No manual steps required. - **Schema migrations:** Database migrations are automatically applied on each deploy. - **Runtime upgrades:** Each build includes the latest Encore runtime. Backward compatibility is maintained across releases. - **Security and IAM:** IAM policies are automatically managed using the principle of least privilege. All data is encrypted in transit and at rest. - **Monitoring and observability:** Built-in distributed tracing, metrics, and logging are available out of the box. - **Cloud provider updates:** The Encore platform handles changes to underlying cloud services as part of normal deployments. ### Under your control - **Infrastructure customization:** Override defaults via the Encore platform dashboard or directly in your cloud provider's console. - **Deploy approval:** Optionally require admin approval for deployments that include infrastructure changes. - **Runtime pinning:** Pin specific runtime or base image versions when you need to. - **PostgreSQL major upgrades:** Major version upgrades are a manual decision, giving you control over timing and testing. - **Disaster recovery tuning:** Encore provides default backups and PITR. You can adjust retention, replication, and HA settings to meet your specific RTO/RPO targets. - **Application-level concerns:** Authentication logic, custom alerting, and domain-specific monitoring are yours to configure as needed. --- Source: https://encore.dev/docs/platform/infrastructure/jit-access Section: Operations # Just-in-time Access > Grant temporary, time-bound deploy permissions instead of standing access Just-in-time (JIT) access lets the Encore platform request the permissions it needs for a deploy **only when they're needed, for a limited time**, instead of holding standing access to your cloud resources. When a deploy starts, the Encore platform requests a temporary privilege grant, uses it to apply the change, and the grant automatically expires afterwards. This minimizes the blast radius of long-lived credentials and gives you a full audit trail of every privileged action: what was requested, why, and for how long. > **Note** > > Just-in-time access is an **Enterprise** feature. [Contact us](https://encore.dev/book) to enable it for your organization. ## Supported clouds | Cloud | Status | | ------- | ---------------------------------------------------------- | | **GCP** | Available, powered by GCP Privileged Access Manager (PAM). | | **AWS** | In active development — [reach out](https://encore.dev/book) if you're interested. | ## GCP On GCP, just-in-time access is powered by [Google Cloud Privileged Access Manager (PAM)][gcp-pam]. Instead of granting the Encore platform's deploy identity permanent IAM roles, you define **entitlements** in GCP PAM that describe which roles can be requested and under what conditions. ### How it works During a deploy, the Encore platform: 1. Requests a grant for the configured entitlement, scoped to the duration it expects to need. 2. Waits for the grant to become active (subject to any approval policies you've configured in GCP PAM). 3. Performs the infrastructure change. 4. Lets the grant expire — access is no longer held once the deploy completes. Because grants flow through GCP PAM, every request is recorded with its justification and duration, giving you a complete audit trail. ### Configuration Just-in-time access is configured **per environment** in the Encore platform dashboard: 1. Open your app and go to **Environment Settings** for the environment you want to configure. 2. Find the **GCP Privileged Access Manager** section. The section has three settings: - **Grant duration** — how long, in minutes, each requested grant stays active. - **Default entitlement** — the GCP PAM entitlement requested for any deploy phase that doesn't have its own override. - **Per-phase overrides** — optional entitlements for specific deploy phases (see below). #### Deploy phases A deploy can involve different kinds of infrastructure changes, and you may want a different entitlement for each. You can override the entitlement on a per-phase basis: | Phase | When it applies | | ------------- | ------------------------------------------------- | | **Provision** | Creating new infrastructure resources | | **Deploy** | Rolling out new application versions | | **Delete** | Removing infrastructure resources | Any phase without an explicit override falls back to the **Default** entitlement. This lets you, for example, require a more privileged (or separately audited) entitlement for `Delete` while everything else uses a standard one. #### Resource overrides By default, every resource in an environment uses the environment-level configuration. When you need finer control — for example, a specific GCP project that requires a different entitlement — you can add a **resource override**. Resource overrides accept the same entitlement settings as the environment default, including per-phase overrides. The **grant duration** is always inherited from the environment default and cannot be overridden per resource. #### Configuration hierarchy Settings are merged **most-specific-first**, so a more specific layer always wins: ``` Resource (project) override ➜ Environment default ➜ App default ``` For a given deploy phase, the Encore platform resolves the entitlement by checking the resource override first, then the environment default, then the app-level default. ### Setting it up 1. In GCP, create the [PAM entitlements][gcp-pam] you want the Encore platform to be able to request, granting the roles needed for deploys (and configuring any approval workflow you require). 2. In the Encore platform dashboard, open the environment's **Environment Settings** and configure the **GCP Privileged Access Manager** section with the entitlement name(s) and grant duration. 3. Trigger a deploy. The Encore platform will request the grant just-in-time and you'll be able to see the activity in your GCP PAM audit logs. ## AWS Just-in-time access for **AWS** is in active development. If you're running on AWS and want time-bound deploy permissions, [reach out](https://encore.dev/book) — we'd love to hear about your use case and can let you know when AWS support is available. ## Related - [Application Security](https://encore.dev/docs/platform/deploy/security) — how the Encore platform secures your application by default - [GCP Infrastructure](https://encore.dev/docs/platform/infrastructure/gcp) — how the Encore platform provisions GCP infrastructure - [Compliance](https://encore.dev/docs/platform/management/compliance) — Encore's own use of time-limited privileged access [gcp-pam]: https://cloud.google.com/iam/docs/pam-overview --- Source: https://encore.dev/docs/platform/infrastructure/external-vaults Section: Operations # External Vaults > Reference secrets stored in external secret stores instead of storing them in Encore By default, the Encore platform stores your application's [secrets](https://encore.dev/docs/ts/primitives/secrets) in the Encore vault and copies them into your cloud provider's secret manager during deploys, so they're available to your application at runtime. **External vaults** let you go a step further and reference secrets that already live in an external secret store, so the secret value never has to be entered into or stored by the Encore platform. This is useful when secrets are owned by another team or system, managed by an existing rotation process, or required by policy to stay in a specific secret store. > **Note** > > External vaults are an **Enterprise** feature. [Contact us](https://encore.dev/book) to enable them for your organization. ## How it works You connect a vault to your Encore app, pointing it at an external secret store. Individual [secrets](https://encore.dev/docs/ts/primitives/secrets) can then be configured to **reference** a secret in that vault — by its name and version — instead of holding a value managed by Encore. At runtime, your application reads the value directly from the external store. ### Supported providers | Provider | Status | | ---------------------- | --------------------------------------------------------------- | | **GCP Secret Manager** | Available. Reference secrets stored in Google Cloud Secret Manager. | | **AWS Secrets Manager** | In active development — [reach out](https://encore.dev/book) if you're interested. | ## Adding a vault Vaults are configured at the **app level**. You need the **Admin** or **Member** (owner/writer) role, and the feature must be enabled for your organization. 1. Open your app and go to **Settings → Secrets**. 2. In the **External Vaults** section, click **Add External Vault**. 3. Choose a **provider** (e.g. GCP Secret Manager). 4. Give the vault a **name** to identify it. 5. Fill in the provider configuration (see below). 6. Click **Save**. ## GCP Secret Manager ### Configuration To connect a GCP Secret Manager vault you'll provide: - **GCP Account** — the connected GCP service account the Encore platform uses to access Secret Manager. If you don't have one yet, [connect a cloud](https://encore.dev/docs/platform/deploy/own-cloud) first. - **Project ID** — the GCP project that hosts the Secret Manager secrets. You then need to grant the connected account access to the secrets. In the GCP project that holds the secrets: 1. Create a custom [IAM role][gcp-iam-roles] with the following permissions: - `secretmanager.secrets.getIamPolicy` - `secretmanager.secrets.setIamPolicy` 2. In [Secret Manager][gcp-secret-manager], grant that role to the connected GCP service account on the secrets you want to reference. The dashboard shows the exact account email and permissions, with copy-to-clipboard helpers, while you're filling in the configuration. Notice that the connected account only needs `getIamPolicy` and `setIamPolicy` — **not** access to the secret values themselves. The Encore platform uses these permissions to grant your application's runtime service accounts access to the referenced secrets; it never needs to read the values itself. ### Preventing Encore from reading secret values Because the connected account holds `setIamPolicy`, it could in principle grant itself (or another principal it controls) access to the secret values. If your security model requires that the Encore platform is *able to manage access but never able to read the secrets*, you can enforce that with two [GCP IAM deny policies][gcp-deny]. Deny policies take precedence over allow policies, so they hold even if an `allow` binding is later added. Attach them at the project, folder, or organization level that contains the secrets. #### 1. Deny secret access outside your production projects This deny policy blocks the `access` permission on Secret Manager for every principal **except** the service accounts that belong to your production project(s). Even if Encore's connected account granted itself an allow binding, this deny rule would override it — only your runtime workloads can read the values. Replace `123456789012` with the numeric project ID of the project whose service accounts are allowed to read the secrets (add an `exceptionPrincipals` entry per production project): ```json { "displayName": "Strict App Project Boundary for Secrets", "rules": [ { "denyRule": { "deniedPrincipals": ["principalSet://goog/public:all"], "exceptionPrincipals": [ "principalSet://cloudresourcemanager.googleapis.com/projects/123456789012/type/ServiceAccount" ], "deniedPermissions": ["secretmanager.googleapis.com/*.access"] } } ] } ``` #### 2. Block Encore from impersonating service accounts The boundary above relies on only your production service accounts being able to read secrets. To prevent Encore's connected account from sidestepping it by impersonating one of those accounts, deny it the Service Account Credentials impersonation permissions. Replace the `deniedPrincipals` entry with the principal identifier of your Encore connected/orchestrator service account: ```json { "displayName": "Block Encore Orchestrator Impersonation Loophole", "rules": [ { "denyRule": { "deniedPrincipals": ["principal://goog/subject/app-1esdad@..."], "deniedPermissions": [ "iam.googleapis.com/serviceAccounts.getAccessToken", "iam.googleapis.com/serviceAccounts.getOpenIdToken", "iam.googleapis.com/serviceAccounts.signBlob", "iam.googleapis.com/serviceAccounts.signJwt", "iam.googleapis.com/serviceAccounts.implicitDelegation" ] } } ] } ``` With both policies in place, the Encore platform can still manage which runtime identities are allowed to read each secret, but cannot read the secret values itself — directly or via impersonation. > **Warning** > > Make sure the `exceptionPrincipals` in the first policy cover every service account your application actually runs as, otherwise your workloads will be denied access to the secret values too. ## AWS External vault support for **AWS** (including AWS Secrets Manager) is in active development. If you're running on AWS and want to reference secrets from an external store, [reach out](https://encore.dev/book) — we'd love to hear about your use case and can let you know when AWS support is available. ## Referencing a vault secret Once a vault is connected, you can point a secret at it instead of storing a value in Encore: 1. Go to **Settings → Secrets** and create or edit a secret. 2. When entering the value, choose the **vault** as the source. 3. Provide the **secret ID** (the secret's name in the external store) and the **version** to reference. Your application keeps reading the secret [by name in code](https://encore.dev/docs/ts/primitives/secrets) — only the source of the value changes. Different environments can reference different vaults or versions, just like regular secrets. ## Managing vaults From the **External Vaults** section you can: - **Rename** a vault. - **Edit configuration** — change the provider configuration (e.g. the GCP account or project ID). - **Remove** a vault. Each vault shows which secrets currently reference it (its "used by" list), so you can see the impact before removing it. ## Related - [Secrets](https://encore.dev/docs/ts/primitives/secrets) — how secrets work in Encore - [Application Security](https://encore.dev/docs/platform/deploy/security) — how the Encore platform secures your application by default - [Connect your cloud account](https://encore.dev/docs/platform/deploy/own-cloud) — connect the GCP account used to access Secret Manager [gcp-iam-roles]: https://cloud.google.com/iam/docs/creating-custom-roles [gcp-secret-manager]: https://cloud.google.com/security/products/secret-manager [gcp-deny]: https://cloud.google.com/iam/docs/deny-overview --- Source: https://encore.dev/docs/platform/infrastructure/manage-db-users Section: Operations # Managing database user credentials The Encore platform provisions your databases automatically, meaning you don't need to manually create database users. However, in some use cases you need access to the database user credentials, so the Encore platform makes it simple to view them. As an application **Admin**, open the [Encore platform dashboard](https://app.encore.dev) and go to the **Infrastructure** page for the relevant environment. In the section for the relevant **Database Cluster**, you will find a **Users** sub-section which lists your database users. Click on the "eye" icon next to each username to decrypt the password. Note that databases hosted in [Encore Cloud](https://encore.dev/docs/platform/infrastructure/infra#encore-cloud-hosting) currently do not expose usernames and passwords. To connect to Encore Cloud-hosted database, use [`encore db shell`](https://encore.dev/docs/ts/primitives/databases#connecting-to-databases). `encore db shell` defaults to read-only permissions. Use `--write`, `--admin` and `--superuser` flags to modify which permissions you connect with. ![View Database User Credentials](https://encore.dev/assets/docs/db-user.png) ## Credential isolation The Encore platform provisions unique credentials at multiple levels to ensure proper security isolation: - Each database instance has its own unique credentials. - Each container connecting to a database uses a unique credential. ## Credential rotation To rotate database credentials, open the [Encore platform dashboard](https://app.encore.dev) and navigate to the **Infrastructure** page for the relevant environment. In the database cluster section, use the rotation controls to generate new credentials. Existing connections will be updated automatically on the next deployment. > **Important** > > Do not change or remove the database users created by Encore, as this will prevent the Encore platform from maintaining and handling connections to the databases in your application. --- Source: https://encore.dev/docs/platform/integrations/github Section: Operations # Integrate with GitHub Encore applications are easy to integrate with GitHub for source code hosting. To link your application to GitHub, open your application in the [Encore platform dashboard](https://app.encore.dev), and click on **Settings** in the main navigation. Then select **GitHub** in the settings menu. Next, connect your account to GitHub by clicking the **Connect Account to GitHub** button. This will open GitHub where you can grant access either to all repositories or only the specific one(s) you want to link with Encore. ![](https://encore.dev/assets/img/git-connect.png) When you come back to Encore, click the **Link App to GitHub** button: ![](https://encore.dev/assets/img/git-begin.png) In the popup, select the repository you would like to link your app with: ![](https://encore.dev/assets/img/git-modal.png) Click **Link** and you're done! Encore will now automatically start building and running tests against your Pull Requests, and on paid plans provision Preview Environments for each Pull Request. Linking doesn't copy your code to the repository. If your code is in Encore's managed git, push it to the GitHub repository, for example with `git remote add origin https://github.com//.git` and `git push -u origin main`. From then on, push to GitHub to deploy. ![](https://encore.dev/assets/img/git-linked.png) ## GitHub Enterprise Cloud Encore supports integration with GitHub Enterprise Cloud (GHEC) for enterprise customers. [Contact us](https://encore.dev/book) for more information. ## Placing your Encore app in a monorepo sub-folder If you already have a monorepo and want to place your Encore application in a sub-folder, you need to tell Encore which folder the `encore.app` file is in. Do this by opening your app in the [Encore platform dashboard](https://app.encore.dev) and go to **Settings** > **General**. Then in the **Root Directory** section, you specify the directory within your Git repository in which your `encore.app` file is located. ## Configure deploy trigger When using GitHub, you can configure Encore to automatically trigger deploys when you push to a specific branch name. To configure which branch name is used to trigger deploys, open your app in the [Encore platform dashboard](https://app.encore.dev) and go to the **Overview** page for your intended environment. Click on **Settings** and then in the section **Branch Push** configure the `Branch name` and hit save. ## Preview Environments for each Pull Request Once you've linked your app with GitHub, Encore will automatically start building and running tests against your Pull Requests. On paid plans, Encore will also provision a dedicated Preview Environment for each pull request. This environment works just like a regular development environment, and lets you test your changes before merging. Learn more in the [Preview Environments documentation](https://encore.dev/docs/platform/deploy/preview-environments). ![Preview environment linked in GitHub](https://encore.dev/assets/docs/ghpreviewenv.png "Preview environment linked in GitHub") --- Source: https://encore.dev/docs/platform/integrations/custom-domains Section: Operations # Custom Domains > Expose APIs from your own domain By default, all application [environments](https://encore.dev/docs/platform/deploy/environments) are accessible as subdomains of the shared Encore domain `encr.app`. When exposing APIs publicly, you often want to provide a URL endpoint branded with your own domain. Follow these instructions to serve your backend using your own custom domain name. This also has the benefit of providing a built-in Web Application Firewall (WAF) using [Cloudflare WAF](https://www.cloudflare.com/en-gb/application-services/products/waf/). ## Adding a domain Modify the DNS records for your domain, adding a CNAME record pointing at: `custom-domain.encr.app` It's recommended to set a TTL (Time-To-Live) of 30 minutes for the CNAME record. > **Important** > > Encore requires that you add a CNAME record for each domain you wish to serve traffic from. > CNAME record using wildcards, e.g. `*.example.com`, are not currently supported. Once you've added the CNAME record, go to the Custom Domains settings page by opening **[Your apps](https://app.encore.dev/) > (Select your app) > Settings > Custom Domains**. Click on `Add Domain` on the top right of the page. Enter the domain name you configured the CNAME on and select which [environment](https://encore.dev/docs/platform/deploy/environments) you wish to serve on that domain, then click `Add`. Encore will now set up your domain and issue SSL certificates to serve traffic through. ![Custom Domain Settings](https://encore.dev/assets/docs/customdomain.png) > **Note** > > If you configure multiple domains against a single environment, Encore will serve traffic through all > configured domains. The `encr.app` subdomain which was created when you originally created an environment will always be > configured to serve traffic to that environment. > > This allows you to migrate to a custom domain safely without risking > cutting traffic off to older clients which may be hard coded to access your applications via the default subdomain. ## Redirecting HTTP to HTTPS Custom domains use the environment's setting for redirecting plain HTTP requests to HTTPS. See [Redirecting HTTP to HTTPS](https://encore.dev/docs/platform/deploy/environments#redirecting-http-to-https) for how to turn it on or off. ## Domain statuses On the Custom Domains settings page, you can see the various statuses throughout the lifecycle of a custom domain. | Status | Description | | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Pending` | The domain is currently queued to be provisioned by Encore. | | `Waiting for CNAME` | Encore is waiting for the CNAME to become active and for the SSL certificate to be issued for the domain. | | `Configuring Edge Routers` | The SSL certificate has been issued and the Encore edge routers are being configured to route traffic on the domain. | | `Active` | The domain is serving traffic to your Encore application. | | `Not Working` | A non-recoverable problem has occurred. This could be a result of the CNAME record being removed or pointed elsewhere. If you see this error, please [contact support](https://encore.dev/contact). | --- Source: https://encore.dev/docs/platform/integrations/webhooks Section: Operations # Webhooks & Events > Set up webhooks to react to Encore events Webhooks provide a way for notifications to be delivered to an HTTP endpoint of your choice whenever certain events happen within Encore. For example, you can set up a webhook to be notified whenever a deployment starts or finishes. Webhooks are defined on a per-application basis, and are configured under Settings -> Webhooks in the [Encore platform dashboard](https://app.encore.dev). To simplify using webhooks, Encore.go provides a Go module, [go.encore.dev/webhooks](https://pkg.go.dev/go.encore.dev/webhooks), that provides type definitions and documentation of all supported webhook events. This module is kept up to date as new events are added. ## Webhook Deliveries Each time an event occurs that matches one of your defined webhooks, Encore will send a HTTP POST request to the webhook's configured URL with information about the event. If the HTTP request fails, the delivery is marked as failed and won't be retried. Each event is given a unique event id, which is shared across all webhooks. Within each webhook, each event is given a sequence number, which is incremented for each event that matches that webhook. The sequence number allows for a linear ordering of events within a webhook, making it easy to determine if an event was missed. These are provided in the `X-Encore-Event-Id` and `X-Encore-Sequence-Id` headers respectively, and are also part of the event payload itself. ## Parsing webhook events To parse a webhook event, use the [`webhooks.ParseEvent`](https://pkg.go.dev/go.encore.dev/webhooks#ParseEvent) function. As you'll see in the example below, to parse the webhook event you'll need access to the webhook secret. This is a secret value that is generated by Encore and is used to sign each webhook request. More about this in the next section. For example, to process rollout started and completed webhook events, you could do something like this: ```go package service import ( "net/http" "go.encore.dev/webhooks" ) var secrets struct { EncoreWebhookSecret string } //encore:api public raw func Webhook(w http.ResponseWriter, req *http.Request) { payload, err := io.ReadAll(req.Body) if err != nil { // ... handle error } event, err := webhooks.ParseEvent(payload, req.Header.Get("X-Encore-Signature"), secrets.EncoreWebhookSecret) if err != nil { // ... handle error } switch data := event.Data.(type) { case *webhooks.RolloutCreatedEvent: // ... handle rollout created event case *webhooks.RolloutCompletedEvent: // ... handle rollout completed event } } ``` > **Note** > > Note that the example above is written as an Encore API endpoint, but that's not required. > The same code works in any Go HTTP server, and the `go.encore.dev/webhooks` library does not depend on > any Encore-specific functionality. ## Checking webhook signatures Since the webhook endpoint is publicly accessible, it is important to validate that the request is coming from Encore. To do so, Encore generates a secret for each webhook, which is used to sign each request. The webhook secret can be found on the webhook details page by admins. If you use the [go.encore.dev/webhooks](https://pkg.go.dev/go.encore.dev/webhooks) library then signature validation is handled automatically, but it's also possible to verify the signature manually (see below). ### Preventing replay attacks A [replay attack](https://en.wikipedia.org/wiki/Replay_attack) occurs when an attacker intercepts a valid request, including the payload and signature, and re-transmits it one or more times, causing unintended side effects. To mitigate such attacks, Encore includes a timestamp in the `X-Encore-Signature` header. This timestamp is part of the signed payload, which means that it can't be changed by the attacker without invalidating the signature. This makes it possible to mitigate replay attacks by ensuring the timestamp isn't older than a certain threshold (the `go.encore.dev/webhooks` library defaults to 5 minutes). ### Verifying signatures manually The `X-Encore-Signature` header included in each webhook event contains a timestamp and one or more *schemes*. The timestamp is prefixed with `t=`, and each *scheme* is prefixed by a `v` and a version number. Currently only the `v1` *scheme* is supported. For example, a valid signature header might look like this: ``` X-Encore-Signature: t=1623345600,v1=0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b ``` The `v1` scheme is using a hash-based message authentication code ([HMAC](https://en.wikipedia.org/wiki/HMAC)) with [SHA-256](https://en.wikipedia.org/wiki/SHA-2). To prevent downgrade attacks, ignore all schemes that are not `v1`. It's possible the signature contains multiple signatures, for example if the webhook secret has been rotated recently. When rotating the webhook secret, Encore lets you define for how long the old secret should continue to be valid for. During that window, each webhook event will be signed with the new and the old secret. To validate the webhook signature, follow the algorithm below: **Step 1: Extract the timestamp and signatures from the header** Split the header on `,` to get a list of fields, then split each field on `=` to get the key and value. The value of the `t` key is the timestamp, and represents the UNIX timestamp (in seconds) when the signature was created. The fields with the `v1` key (possibly several, in case of secret rotation) are the signatures. Discard any other fields. **Step 2: Prepare the payload to sign** Create the payload to sign by concatenating the timestamp (as a string) and the request body, separated by `.` like so: ```go payloadToSign := timestamp + "." + string(payload) ``` **Step 3: Compute the expected signature** Compute the HMAC with the SHA256 hash function, using the webhook secret as the key and the `payloadToSign` as the message. Then, encode the resulting HMAC using the base64 URL encoding, and trim any trailing `=` characters. In Go, this can be done like so: ```go h := hmac.New(sha256.New, []byte(webhookSecret)) h.Write([]byte(payloadToSign)) digest := h.Sum(nil) expectedSignature := base64.RawURLEncoding.EncodeToString(digest) ``` **Step 4: Compare the signatures** Compare each signature with the `v1` field in the header with the expected signature. To protect against timing attacks, use a constant-time comparison function (like `crypto/hmac.Equal` in Go). If none of the signatures match, reject the request. If a match is found, compare the timestamp with the current time. If the difference is greater than the allowed threshold (5 minutes is a reasonable default), reject the request. Otherwise, accept the request. --- Source: https://encore.dev/docs/platform/integrations/oauth-clients Section: Operations # OAuth Clients OAuth clients provide a framework for delegated and scoped access to the Encore platform API. An OAuth client creates short-lived access tokens on demand, and supports the principle of least privilege by allowing fine-grained control on the access granted to the client using scopes. ## How it works You create an OAuth client that defines the scopes to allow when your client application uses the Encore platform API. Scopes are currently grouped into "roles", which include a set of permissions. For example, the `deployer` role grants access to the triggering deployments. An OAuth client consists of a client ID and a client secret. When you create an OAuth client, the Encore platform creates these for you. Within your client application, use the client ID and client secret to request an API access token from the Encore platform's OAuth token endpoint. You use the access token to make calls to the Encore platform API. The access token grants permission only for the scopes that were defined when you created the OAuth client. An API access token expires after one hour. For continuous access, shortly before an API access token expires, request a new API access token from the Encore platform's OAuth token endpoint. OAuth client libraries in popular programming languages can handle the API access token generation and renewal. The Encore platform's OAuth implementation is based on the [OAuth 2.0 protocol](https://www.rfc-editor.org/rfc/rfc6749). ## Prerequisites You need to be an Owner of the Encore application in order to create or revoke OAuth clients. ### Setting up an OAuth client Open the OAuth clients page in the application settings page. In the Generate OAuth client dialog, select the set of operations that can be performed with tokens created by the new OAuth client. After generating the client, you can see the new OAuth client's ID and secret. Copy both the client ID and secret, as you need them for your client code. Note that after you close the Generated new OAuth client dialog, you won't be able to copy the secret again. **Store the client secret securely.** Your OAuth client is now configured. Use the client ID and secret when you configure your OAuth client application. Note that the provided client secrets are case-sensitive. If an OAuth client is created by a user who is later removed from your application, the OAuth client will continue to function and generate API access tokens. Application owners can see all configured OAuth clients in the OAuth clients page of the application settings. ### Roles Roles define which operations are permitted in API access tokens that are created by your client application. Currently there is a single supported role: **Deployer**. The deployer role allows for programatically triggering deployments. When new the Encore platform functionality is provided, we will add it to existing roles where applicable. That means a role is not restricted to only access of APIs that existed at the time the client was initially authorized — a role will contain additional access where it makes sense for new or updated functionality. ### Revoking an OAuth client Open the OAuth clients page of the application settings page. Find the OAuth client that you want to delete and select Revoke. Select Revoke OAuth client to confirm you want to revoke the OAuth client. When you revoke an OAuth client, any active API access tokens that were created by the client are also revoked. ### The Encore platform OAuth token endpoint The Encore platform's OAuth token endpoint is https://api.encore.dev/api/oauth/token. See the [Encore platform API Reference](https://encore.dev/docs/platform/integrations/api-reference) documentation for more information. Make requests to the OAuth token endpoint when you need an API access token. The OAuth token endpoint accepts requests that conform to the OAuth 2.0 client credentials grant request format, and returns responses that conform to the OAuth 2.0 client credentials grant response format. ## OAuth client libraries Popular programming languages provide OAuth client libraries to simplify your use of OAuth clients. For example, the following Go code shows how to create an OAuth client object that uses your client ID and client secret to generate an API access token for calls to the Encore platform API. Similar libraries exist for other popular programming languages. ```go package main import ( "context" "os" "golang.org/x/oauth2/clientcredentials" ) func main() { oauthConfig := &clientcredentials.Config{ ClientID: os.Getenv("OAUTH_CLIENT_ID"), ClientSecret: os.Getenv("OAUTH_CLIENT_SECRET"), TokenURL: "https://api.encore.dev/api/oauth/token", } client := oauthConfig.Client(context.Background()) // Make API calls using `client.Get` etc. resp, err := client.Get("https://api.encore.dev/api/...") // ... } ``` The example requires that you define environment variables `OAUTH_CLIENT_ID` and `OAUTH_CLIENT_SECRET`, with their values set to the client ID and client secret that are created when you set up an OAuth client. ### Verifying you can generate API access tokens After you set up an OAuth client, an easy way to confirm that you can generate API access tokens is to make a curl request to the Encore platform OAuth token endpoint. ```bash curl -d "client_id=${OAUTH_CLIENT_ID}" -d "client_secret=${OAUTH_CLIENT_SECRET}" \ -d "grant_type=client_credentials" "https://api.encore.dev/api/oauth/token" ``` The example requires that you define environment variables OAUTH_CLIENT_ID and OAUTH_CLIENT_SECRET, with their values set to your client ID and client secret. Here's an example response showing the API access token: ```json {"access_token":"MTcyODQ3NTg3NXww...GDxfmxnuq9zDEAmHmP5D44=","token_type":"Bearer","expires_in":3600, "actor": "o2c_my_key_id"} ``` ## Limitations An OAuth access token expires after 1 hour — this duration cannot be modified. --- Source: https://encore.dev/docs/platform/integrations/auth-keys Section: Operations # Generating Auth Keys Pre-authentication keys (“auth keys” for short) let you authenticate the Encore CLI without needing to sign in via a web browser. This is most useful when setting up CI/CD pipelines. ## Types of auth keys - **Reusable Keys** for authenticating more than one machine. - **Ephemeral Keys** - Machines authenticated by this key will be automatically logged out after one hour. > **Important** > > **Be very careful with reusable keys!** These can be very dangerous if stolen. They're best kept in a key vault product (eg. 1Password, LastPass) specifically designed for the purpose. ## Authentication **Auth keys** authenticate a machine as the Encore app for which the key was generated. If Ada generates an auth key, and uses it to set up her CI/CD pipeline, then that machine is authenticated as Ada's Encore app. ## Generating a key ### Step 1: Generate an auth key As an Encore user, visit the auth key page by going to **[Your apps](https://app.encore.dev/) > (Select your app) > App Settings > Auth Keys**. A key can be both **reusable** and **ephemeral** at the same time (you can decide the combination based on your particular use case). > **Important** > > **Don't forget to store your key!** Once generated, you will need to copy and store your key in a vault product (eg. 1Password, LastPass). We do not display the full contents of a key in our dashboard for security reasons. This page also gives you the ability to revoke existing keys. ### Step 2: Authenticate with the auth key Using the Encore CLI, you can authenticate with your newly generated key: ```shell $ encore auth login --auth-key=ena_nEQIkfeM43t7oxpleMsIULbhbtLAbYnnLf1D ``` ## Revoking a key You can revoke a key simply by pressing the **Delete** button next to it. This will prevent any machines currently using it to authenticate to the Encore platform (regardless of the key type). --- Source: https://encore.dev/docs/platform/integrations/terraform Section: Operations # Terraform Provider > Integrate Encore with existing infrastructure Encore makes it simple to deploy and manage cloud applications. When you're dealing with a large and complex system, you may want to integrate Encore-provisioned resources with an existing infrastructure landscape. For this purpose, Encore maintains a Terraform Provider with data sources for all Encore-provisioned resources. ## Understanding Encore Terraform Data Sources Encore Terraform data sources act as read-only references to resources Encore has already provisioned on your behalf. Unlike Terraform resources (which create or modify infrastructure), data sources only retrieve information. The Encore data sources let's you retrieve cloud identifiers for resources managed by Encore, such as databases, caches, and more. To do this, you only need to provide the name of the resource and the environment it's in. ## Configuring the Encore Terraform Provider To use Encore data sources, you need to declare the Encore Terraform provider in the `required_providers` of your Terraform configuration file. Here's an example of how to declare the provider: ``` terraform { required_providers { encore = { source = "registry.terraform.io/encoredev/encore" } } } ``` Once you've declared the provider, Terraform will automatically download the provider plugin when initializing the working directory using `terraform init`. To authenticate with the Encore API, the provider need an Encore Auth Key. You can generate an auth key from Encore's [Cloud Dashboard](https://encore.dev/docs/platform/integrations/auth-keys). Once you have the auth key, you can configure the provider in your Terraform configuration file like this: ``` provider "encore" { env = "your-env" auth_key = "your-auth-key" } ``` You can also set the `ENCORE_AUTH_KEY` environment variable to avoid hardcoding the auth key in your configuration file. ## Using Encore Terraform Data Sources Once you have the provider configured, you can use the Encore data sources to retrieve information about resources. There are several data sources available, such as `encore_database`, `encore_cache`, and `encore_pubsub_topic`. Each data source has its own set of attributes that you can use to retrieve information about the resource. The full documentation for each data source is available in the [Terraform Registry](https://registry.terraform.io/providers/encoredev/encore). Here's an example of how to use the `encore_pubsub_topic` data source to connect AWS IOT Core to an Encore PubSub topic: ``` data "encore_pubsub_topic" "topic" { name = "my-topic" env = "my-env" } resource "aws_iot_topic_rule" "rule" { name = "my-rule" sql = "SELECT * FROM 'my-topic'" sns { message_format = "RAW" role_arn = aws_iam_role.role.arn target_arn = data.encore_pubsub_topic.topic.aws_sns.arn } } ``` --- Source: https://encore.dev/docs/platform/management/compliance Section: Management & billing # Security & Compliance > How Encore protects your applications, code, and data _Last updated: June 24, 2026_ Your applications, code, and data are among your most important assets. Security is foundational to everything we build at Encore — it is embedded in our architecture, our processes, and our culture. This document provides a comprehensive overview of the security controls and practices we have in place today, structured around the SOC 2 trust service criteria. ### Security at a glance | Area | What we do | | --- | --- | | **Infrastructure** | Hosted on GCP (ISO 27001 / SOC 2 certified). All servers are private, accessible only via VPN. | | **Encryption** | AES-256 at rest, TLS 1.2+ and WireGuard in transit. Customer secrets additionally encrypted via GCP KMS. | | **Zero-trust networking** | All server-to-server communication is authenticated and end-to-end encrypted via Tailscale / WireGuard. | | **Access control** | Principle of least privilege, MFA enforced, regular access reviews, VPN-only infrastructure access. Time-limited privileged access via GCP Privileged Access Manager (PAM) with full audit trail. | | **Authentication** | End-user authentication managed by Clerk (SOC 2 certified). Passwordless by default — Encore never stores or handles user passwords. Employee access governed by Google Workspace SSO with mandatory MFA. | | **Monitoring & alerting** | 24/7 monitoring via Grafana, Sentry, Cronitor, and GCP Cloud Monitoring (all SOC 2 certified). | | **Vendor security** | All critical vendors are SOC 2 and/or ISO 27001 certified (GCP, Tailscale, Clerk, GitHub, Sentry, Grafana). Vendors are risk-classified and reviewed under a formal Vendor Management Policy. | | **Code quality** | Mandatory code review, CI/CD with automated testing, automated vulnerability scanning. Governed by a formal Change Management Policy. | | **Documented policies** | Formal written policy suite in place: Information Security, Vulnerability Management, Incident Response, Change Management, Data Classification, and Vendor Management. | | **Data privacy** | GDPR compliant. Data minimization by design. Four-tier data classification (Public, Internal, Confidential, Restricted). | | **Responsible disclosure** | Active bug bounty program for security researchers. | ## SOC 2 SOC is short for "System and Organization Controls" — it is the de facto industry standard for software security and privacy. We have implemented controls aligned with the SOC 2 framework and are actively preparing for a formal SOC 2 Type I audit, during which an external auditor will verify that our controls meet the standard. As part of this preparation, we have formalized our security program into a written policy suite, including an Information Security Policy, Vulnerability Management Policy, Incident Response Policy, Change Management Policy, Data Classification Policy, and Vendor Management Policy. These documents codify the practices described on this page and are available to customers under NDA as part of a vendor review. Our current SOC 2 timeline is: - **SOC 2 Type I:** target completion Q4 2026. - **SOC 2 Type II:** target completion Q2 2027, with the observation period beginning immediately after Type I. - **Third-party penetration test:** target completion October 30, 2026, so the report is available within the Type I audit window. An executive summary and remediation status are available without NDA; the full report is available under NDA on request. After the Type I audit, we will proceed to Type II, which involves continuous monitoring of our controls over an extended observation period. ### Trust Service Criteria The five SOC 2 trust service criteria are: security, availability, confidentiality, processing integrity, and privacy. **1\. Security** Protecting systems against unauthorized access. **2\. Availability** Ensuring that the system remains functional and usable. **3\. Confidentiality** Restricting the access of data to a specified set of persons or organizations. Ensuring that network communication is encrypted and cannot be intercepted by unauthorized personnel. **4\. Processing integrity** Ensuring that a system fulfills its purpose and delivers correct data. **5\. Privacy** Minimal processing and use of personal data in accordance with the law. The following sections describe in detail how Encore implements each trust service principle. ## Security We believe security is achieved through proven best practices and industry standards — not through obscurity or homegrown cryptography. Our approach is defense in depth: multiple overlapping layers of protection so that the compromise of any single layer does not result in a breach. We have a designated Security Officer responsible for all aspects of security across infrastructure, software, and data. ### Infrastructure security Encore's core production infrastructure is hosted on GCP (Google Cloud Platform), an ISO27001/SOC 2 compliant vendor. Auxiliary services are provided by Hetzner, an ISO27001 compliant vendor. Tailscale, a SOC 2 compliant vendor, provides VPN (Virtual Private Network) services used to secure communication between all servers. All core data processing is carried out in the US East region (us-east-1), and backups are kept in multiple separate regions in the US. Each region is composed of at least three "availability zones" (AZs) which are isolated locations, designed to take over in case of a catastrophic failure at one location. AZs are separated by a significant distance such that it is unlikely that they are affected by the same issues such as power outages, earthquakes, etc. Physical access to GCP is restricted by GCP's security controls. Furthermore, GCP monitors and immediately responds to power, temperature, fire, water leaks, etc. Access to Encore's production infrastructure is restricted to Encore employees. All systems have access controls and only a limited number of employees have privileged access. Access is only possible through a VPN over Tailscale. The production environment is separated from testing environments, using separate accounts and VPCs (Virtual Private Cloud) in GCP. This ensures that any defect in a test environment cannot impact the production system. The connection to the internet is controlled by dedicated gateways. ### Organizational security Since an organization is only as good as its people, Encore takes great care when selecting and training its staff. All employees undergo a thorough selection process that has been designed to identify the best talent in the world for the job. Many of Encore's employees have extensive experience working in regulated environments such as Online Banking and large-scale Online Payments. Individual performance monitoring is carried out by managers on a bi-weekly cadence. Overall organizational performance is tracked continuously and reviewed by management on a monthly cadence using Key Performance Indicators determined by management. Encore is rolling out mandatory annual security awareness training for all employees and contractors, to be in place ahead of the SOC 2 Type I audit in Q4 2026. The training covers security responsibilities, data handling, access protection, phishing and social engineering, device security, incident reporting, and the relevant company security policies. It is designed to increase sensitivity to physical security (hardware and media handling, office access control, etc.), digital security (e.g. secure passwords, two-factor authentication), social engineering attacks ("phishing"), and other security-related topics. Encore employment policy mandates full-disk encryption on all employee devices. ### Product security Encore is aware of how important it is for its customers that all data is handled securely. Therefore, several layers of protection ensure that the data is not accessible to unauthorized persons. An essential part of software security is "defense in depth" which means that there are multiple layers of protection. In case one layer is breached, the next layer helps to contain the breach and mitigate its consequences. This can be achieved by isolating software components from each other, such that the breach of one component does not affect adjacent software. Encore's service-based architecture provides natural isolation between components, and we have adopted a zero-trust security model with the use of Tailscale. All server-to-server communication is authenticated and end-to-end encrypted with WireGuard. GCP's VPC (Virtual Private Cloud) provides another layer of isolation from the internet on the network level. None of Encore's servers are publicly accessible on the internet. As a general principle, all of Encore's data is encrypted while being transported across networks and when stored ("in transit and at rest"). In case of unauthorized access to the data, an attacker would only see undecipherable garbage which cannot be decrypted without the corresponding keys. The encryption methods employed by Encore are industry standard and deemed unbreakable by contemporary standards. Data at rest (virtual filesystems, relational databases, and object storage) is encrypted using GCP's industry-standard AES-256, while data in transit is encrypted with TLS ≥ 1.2 (for Encore's REST API) or WireGuard (for internal communication). All customer secret information is further encrypted using GCP's Key Management Service (KMS). Any access to encrypted data by Encore employees requires elevated access and approval by multiple parties, and all such activity is audited. Elevated access to production systems is granted on a time-limited basis through GCP's Privileged Access Manager (PAM), which provides an audit trail of every request, its purpose, and its duration. Production database access does not expose cryptographic keys or secrets, which remain secured in GCP Cloud KMS. User account authentication is provided by _Clerk_, a SOC 2 compliant vendor. There are two ways for a user to log in to Encore: Single sign-on (SSO) and username plus password. Single sign-on can be used by organizations to fully manage access to Encore and, for example, ensure that former employees no longer have access after the offboarding period. Encore supports Google and GitHub SSO using OAuth. If no SSO is used, the default login method is passwordless login using email and "magic link", also handled by _Clerk_. Encore does not store or in any way handle passwords, neither in plaintext nor cryptographic hash form. This means that Encore does not know the passwords of any users, and no passwords can be reconstructed from our databases. Encore offers bug bounty incentives to individuals who discover any security discrepancies. The objective of offering bug bounty incentives is to receive security-related bug reports from trusted "white hat hackers" before the vulnerability is actively exploited in a malicious way. This contributes to maintaining Encore's product security. All security issues undergo a triaging process by Encore's designated Security Officer and are escalated based on their criticality. Encore uses automated scans to detect software vulnerabilities. All teams are continuously monitoring their services for vulnerabilities and are committed to pro-actively reducing them. The progress is supervised by the Security Officer. Our Vulnerability Management Policy defines how findings are identified, prioritized based on actual risk (not scanner severity alone), and tracked to a clear outcome. Material findings are reviewed by a human, and each is fixed, mitigated, determined not applicable, deferred with a documented rationale, or accepted as a known risk. We target the following remediation timelines: | Severity | Remediation target | | --- | --- | | Critical | 7 days, or faster if actively exploited | | High | 30 days | | Medium | 90 days | | Low | Best effort / backlog | These targets are guided by judgment: a critical issue under active exploitation may warrant same-day mitigation, while a high-severity finding in unused code may be downgraded or closed as not applicable. ### Access control We regularly keep track of and review the list of employees who have access to which systems and remove access where applicable to ensure least access principles apply. Offboarding processes ensure that former employees cannot access internal systems anymore after the termination of their contract. Thanks to the VPN, Encore can centrally restrict access to internal networks. #### MFA Multi-factor authentication (MFA) adds another layer of security on top of classic password authentication. In addition to username and password, the user requires another individual token of access. Stealing or guessing the password is not enough for an attacker to gain access to a system, because the second factor would also need to be stolen. Usually, the second factor is a physical device, such as a mobile phone which has been paired with the authentication system. Encore employs MFA to protect access to the infrastructure provider (GCP) and the version control systems (GitHub), among other systems. Employee access to internal systems is governed by Google Workspace SSO, configured to require mandatory MFA for all employees and to enforce strong password requirements. SMS-based factors are disabled in favour of stronger methods due to SIM-swap and spoofing risks. Server (SSH) access is available only over the Tailscale VPN, and obtaining `sudo` or root access requires re-authenticating with our SSO provider. Employee endpoints are used for development only and do not have standing access to sensitive production systems; elevated access requires Tailscale or time-limited elevation through GCP Privileged Access Manager. ### Vendor and third-party risk management Encore relies on third-party services to build and operate its platform, and we manage the risk they introduce through a formal Vendor Management Policy. We maintain an inventory of vendors used for production systems, customer support, engineering, security, and other business-critical functions. Each vendor has a designated internal owner responsible for ensuring it is reviewed, used for its intended purpose, and offboarded when no longer needed. Vendors are reviewed in proportion to the risk they introduce, rather than with a single one-size-fits-all process. A vendor is classified as **Critical** when its failure or compromise could materially affect our ability to serve customers, protect customer data, or operate production systems — for example cloud infrastructure providers, identity providers, and platforms with administrative access to production. **High-risk** vendors have sensitive access or data but are not essential to running the service, and **Medium** and **Low-risk** vendors have progressively more limited access. Before adopting a Critical or High-risk vendor, we assess what data and systems it can access, what happens if it is breached or unavailable, and whether it provides appropriate security evidence such as a SOC 2 report, ISO 27001 certificate, or penetration test summary. Data protection terms (such as a DPA) are put in place where personal data is processed, access is limited to the minimum necessary, and SSO with MFA is required for administrative access. Critical and High-risk vendors are reviewed at least annually, and vendor access is removed and documented on offboarding. All of Encore's current critical vendors (including GCP, Tailscale, Clerk, GitHub, Sentry, and Grafana) are SOC 2 and/or ISO 27001 certified. ## Availability Hosted on a cloud infrastructure, Encore implements a service-based architecture where many dedicated software components operate isolated from one another, but in a coordinated way, much like a complex machine where individual parts can be replaced independently from one another. During the release of a new version of Encore services, Encore's engineers take great care during the preparation of the update so that in case of an unexpected problem, the system can be restored to the previous state in a manner that minimizes user impact. ### Performance monitoring Encore uses a number of performance monitoring systems, such as Sentry, Cronitor, Grafana, and Google Cloud Monitoring (all being SOC 2 compliant vendors). Grafana is used to monitor application performance, such as server response times and user interface speed. Grafana also collects server-side metrics like CPU and RAM usage. Additionally, Encore monitors the performance of databases with GCP tooling. Slack, a SOC 2 compliant vendor, is used as the alerting channel to notify the developers in case the performance of the system has regressed, for example, due to increased response times, or increased error rates. To enable root cause analysis of bugs, Encore collects system logs from all parts of the system. These logs can only be accessed by authorized users. Encore offers a public "Status page" where users and customers can find the current status of Encore systems. It is available at: [https://status.encore.dev/](https://status.encore.dev/). ### Backups and disaster recovery To reduce the risk of simultaneous failure, Encore backs up data to multiple US regions in GCP, with very limited access. Relational databases are backed up on a daily schedule. Encore will conduct its first full disaster recovery exercise in Q3 2026, cloning the production environment from backups end-to-end and validating its soundness. The exercise covers restoring the primary relational databases and object storage, end-to-end validation against our documented recovery objectives (an 8-hour Recovery Time Objective and a 24-hour Recovery Point Objective), and a tabletop walk-through of our business continuity plan for personnel and communications. DR testing will then run at least annually, plus on any material architecture change. Outcomes and remediation items are recorded and available for SOC 2 audit and, under NDA, for customer review. ### Incident handling Whenever an incident occurs, Encore's designated on-call engineer initiates an investigation and escalates to the broader engineering team as necessary based on severity. Our Incident Response Policy defines escalation paths and a severity model (Low, Medium, High, and Critical) that distinguishes unverified suspicions from confirmed, actively exploited risks. For issues deemed critical, the response team follows an iterative process: a war room is convened, a breach timeline and indicators of compromise are maintained, emergency and long-term mitigations are tracked, and the incident is driven to a documented post-mortem. High-severity incidents require a retrospective. Critical incidents escalate directly to the CEO, CTO, and COO, and engage legal and PR where breach notification may be required. Customers and users can report outages via regular support channels (for example via email, or using the [Discord](https://encore.dev/discord) chat group). Encore's internal communication systems have dedicated channels for incident escalation, with out-of-band communication arrangements in case primary channels are compromised. ## Confidentiality When you use Encore, other users won't be able to see your content, unless you grant access explicitly by inviting them to your application. Encore engineers may use your data to provide support and when necessary to fix bugs. ### Data classification Our Data Classification Policy defines four sensitivity levels, each with corresponding handling requirements: - **Public:** information approved for public release, such as published documentation and marketing content. - **Internal:** information intended for company use that is not harmful if disclosed in small amounts. - **Confidential:** information that could harm customers, the company, employees, or partners if disclosed. Customer production data, private source code, and support tickets default to Confidential. Access requires a business need, is protected with SSO, MFA, and role-based access, and is encrypted in transit and at rest. - **Restricted:** information where disclosure could cause serious harm, such as production secrets, tokens, private keys, and credentials. Access is limited to specific authorized people or systems, stored only in approved secret managers, and rotated if exposure is suspected. Access to internal support tooling is curated so that it does not expose Confidential customer data, is available only over the Tailscale VPN with SSO authentication, and logs all actions for audit. Access to Confidential and Restricted data is granted on a business-need basis and removed promptly on role change, offboarding, or when the need ends. ### Access controls All employees and contractors are contractually bound to confidentiality, which persists after the termination of the work contract. As part of a "clean screen" policy, all computers used by Encore staff must be set to automatically lock the screen after 1 minute of inactivity. All systems access is subject to the "principle of least privilege", meaning that every employee only has access to the systems necessary to perform their official duties. ### Deletion User data will be stored by Encore after the termination of a subscription term, according to [Encore's Terms of Service](https://encore.dev/legal/terms). When a user requests the deletion of data, the data is made inaccessible or physically deleted, depending on the data type and storage location. For technical reasons, data may remain in backups after this point. ## Processing integrity ### Quality assurance Product quality is very important to Encore. There are many different facets, including: - Accuracy and usability of services provided - High performance of the user interface and Encore services - Almost zero downtime Several measures are put into place in order to keep product quality high: - Code review: Code changes are reviewed by a peer of the developer before it is accepted into the main code branch (for critical systems) or in a weekly post-hoc review process (for auxiliary systems). For critical systems code can only be merged if the reviewer agrees. For auxiliary systems any requested changes by reviewers are made as part of the review process. It is often necessary to add a test alongside, and the code review process ensures that this has been done as well. - Continuous integration: Before code is accepted, it is built by our continuous integration environment and tests are executed. If the build fails, the developer is notified immediately and a fix is required before the code can be merged. - Manual testing: Once the code has been merged, the change is deployed and in most cases tested manually post-release to verify quality in the production environment. - Automated integration testing: A large battery of automated tests is executed against the local and production environments and checks many common workflows for regressions of any kind. In case the tests fail, the engineer will address the issues before proceeding with attempting to merge again. - Testing of Open-Source libraries: Encore uses Open-Source libraries to provide certain functionality. Overnight tests run daily to discover potential issues, and manual testing is performed when any Open-Source libraries are version updated. Any code change is released only if all these steps succeed. Furthermore, access to the code base is protected via multi-factor authentication (MFA), which poses another layer of defense against the malicious injection of code. These practices are governed by our Change Management Policy, which requires that production changes go through GitHub pull requests, are reviewed by someone other than the author (segregation of duties enforced via protected branches), pass required automated checks before merge, and are deployed through approved CI/CD workflows. The pull request, review history, CI results, and deployment record together form the audit trail. Higher-risk changes — such as those affecting authentication, cryptography, secrets, billing, network exposure, or destructive database migrations — require additional approval from an appropriate owner and, where applicable, a documented rollback or recovery plan. Emergency changes may be expedited to contain an incident but must be documented and reviewed afterward. Since Encore depends on third-party software, we regularly contribute to the quality assurance of our suppliers. Whenever Encore becomes aware of regressions or bugs, they are reported upstream. In this way, Encore is contributing to the quality, stability, and accuracy of other software in the space. ### Process monitoring Where possible, Encore uses software to enforce processes. For example, code review and having tests passed are enforced by the source control management tool GitHub. Regular reviews on different levels (individual, team, company) foster alignment between all individuals and the company objectives. ### Privacy Encore takes data privacy very seriously and complies with the rules of the European Union's GDPR (General Data Protection Regulation). GDPR grants a wide range of rights to Encore's users, such as the right to be informed, the right to access, the right to rectification, the right to erasure, and others. One fundamental rule of the GDPR is the principle of "data minimization", which ensures that we are not processing more personal data than necessary. As a result, the Encore platform uses only minimal personal data for user authentication and essential communication (a name and contact email). As described above, Encore does not store or handle passwords in any form. ### Privacy policy We are aware that confidential handling of your data is essential to establishing trust. Therefore, [Encore's Privacy Policy](https://encore.dev/legal/privacy) ensures that the data of our users is protected according to the high standards of GDPR. ## Questions and further information We are committed to transparency in our security practices. If you have questions about our security or compliance posture, or would like to request additional documentation for your vendor review process, please contact us at [hello@encore.dev](mailto:hello@encore.dev). --- Source: https://encore.dev/docs/platform/management/billing Section: Management & billing # Plans & Billing Encore offers a **Free** plan for teams that want a simple development workflow and collaboration features. If you want access to all features and want to use Encore's DevOps automation tools for AWS & GCP, there's a paid **Pro** plan available. See the [pricing page](https://encore.dev/pricing) for a feature comparison between each plan and complete pricing information. ## When do I need to upgrade to a paid plan? The Free plan comes with certain limitations, and should your needs exceed one or more of them, you may wish to upgrade to a paid plan: - When you need more than 2 cloud environments - When you want to [automate DevOps in your cloud on AWS/GCP](https://encore.dev/docs/platform/infrastructure/infra) - When you want [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments) for each Pull Request - When you want to use a Custom Domain - When you need multiple concurrent builds - When you need guaranteed logs & tracing retention - When you want access to private support & onboarding assistance - When you want custom configuration for environments hosted on Encore Cloud There is a free 14-day trial of the Pro plan, available to all new Encore users. It's a great way to try out all the features and learn if the Pro plan suits your needs. You can activate your trial from the [pricing page](https://encore.dev/pricing). ## Do I need to pay for hosting? All plans come with free use of Encore Cloud, subject to [Fair Use limits](https://encore.dev/docs/platform/management/usage). Encore Cloud is intended for development environments and limited scale professional use that does not require specific SLAs. For production use, you can [connect your own cloud account on AWS/GCP](https://encore.dev/docs/platform/deploy/own-cloud) and use Encore to deploy there, including provisioning infrastructure and managing IAM. When you connect your own cloud account, you pay for usage directly to your cloud provider. If you prefer to manage deployment yourself, you can export your application as a standalone Docker image and deploy in any way you prefer. ([Learn more](https://encore.dev/docs/self-host/build)) ## Payments & Billing FAQ ### What is the price of the paid plan? See the [pricing page](https://encore.dev/pricing) for complete pricing information. If you are a large organization with specific needs, please [email us](mailto:hello@encore.dev) or [book a 1:1](https://encore.dev/book) to discuss your needs and get a custom price quote. ### Can I pay with a credit card? Yes, we offer payments via Stripe using all major credit cards. You can upgrade your account via the [pricing page](https://encore.dev/pricing). ### What happens if my payment fails? If your payment fails, everything will keep working as normal! We will reach out to you with instructions on how to update your payment information so that we can try to process your payment again. We will not downgrade your account without prior notice. --- Source: https://encore.dev/docs/platform/management/permissions Section: Management & billing # Roles & Permissions > For teams building applications together Encore applications have three membership roles with different permissions: **Admins**, **Members**, and **Viewers**. Here is a breakdown of the key differences between each role: | | Admins | Members | Viewers | | ----------------------------------- | ------ | ------- | ------- | | Manage team members | Y | N | N | | Connect/Disconnect cloud accounts | Y | N | N | | Integrate with GitHub | Y | N | N | | Configure custom domains | Y | N | N | | Manage environments | Y | N | N | | Create auth keys | Y | N | N | | Approve infrastructure provisioning | Y | N | N | | Delete applications | Y | N | N | | Push code changes | Y | Y | N | | Create builds & deployments | Y | Y | N | | Configure secrets | Y | Y | N | | Pull secrets | Y | Y | Y | | Run locally | Y | Y | Y | | View API documentation | Y | Y | Y | | View Architecture | Y | Y | Y | ### Admins Admins have full privileges and can administer your entire application. ### Members Members are active contributors to your applications. They are able to do everything that is not limited to Admins. ### Viewers Viewers are read-only members. They can view the Cloud Dashboard and run your application locally. This role is intended for any team members not contributing directly to your Encore application, but who still get value from certain access. In a bigger team, this role is often appropriate for e.g. Frontend developers and Product Managers. ### Custom roles & permissions Custom roles & permissions is an optional add-on to the [Pro plan](https://encore.dev/pricing), please [contact us](mailto:hello@encore.dev) to discuss your requirements. --- Source: https://encore.dev/docs/platform/management/usage Section: Management & billing # Usage limits > The Encore platform limits and Fair Use guidelines Encore comes with a built-in development cloud, the Encore platform, that is free to use for development and limited scale commercial projects without any specific SLA requirements. The Encore platform is subject to Fair Use guidelines and comes without warranty, as it's not intended for large-scale business-critical use cases. For production use cases, Encore is designed to be used together with your cloud on the major cloud providers (AWS/GCP), and provides full DevOps automation for deployments to your own cloud account. This means Encore has no incentive to increase your usage – rather we can focus on building tools to help you minimize your cloud spending! (Should you wish to use the Encore platform instead of your own cloud account, please [contact us](https://encore.dev/book).) When you use Encore together with AWS/GCP, you can still use the Encore platform to host Preview Environments and development environments. ### Examples of Fair Use - Prototyping & development - Hobby projects - Commercial use cases that have limited load and do not require any SLAs ### Never Fair Use - Proxies and VPNs - Media hosting for hot-linking - Scrapers - Crypto Mining - CPU-intensive APIs (e.g.: Machine Learning) - Load Testing ## Usage guidelines We expect most users to fall within the usage limits below. We want to be as flexible and permissive as possible, and will wherever possible reach out and work with you to find a good solution should we notice that you are exceeding these limits. For users on a [paid plan](https://encore.dev/pricing), we can change these limits to support your needs. If you have significantly higher requirements, this may come with an additional charge (at cost) to cover the extra capacity. Please [contact us](https://encore.dev/book) for more information. **We will never charge you for usage of the Encore platform unless expressly agreed in advance.** ### Usage limits | | Per application | | ---------------- | --------------- | | Requests | 100,000 / day | | Database Storage | 1 GB | | PubSub Messages | 100,000 / day | | Cron Jobs | Once every hour | | Object Storage | 1 GB | ### What happens if I reach a usage limit? If your application reaches a usage limit, we will **not** automatically stop it. Our team will reach out to you, and work with you to find a solution that does not cause any undue disruption to your application. --- Source: https://encore.dev/docs/platform/management/telemetry Section: Management & billing # Telemetry Telemetry helps us improve the Encore by collecting usage data. This data provides insights into how Encore is used, enabling us to make informed decisions to enhance performance, add new features, and fix bugs more efficiently. Encore only collects telemetry data in the local development tools and in the Encore platform dashboard. It does **not** collect any telemetry data from your running applications or cloud services, ensuring complete privacy and security for your operations. ## Why We Collect Data We collect telemetry data for several important reasons: 1. **Improvement of Features**: Understanding which features are most used helps us prioritize improvements and new feature development. 2. **Performance Monitoring**: Tracking performance metrics enables us to identify and resolve issues, ensuring a smoother user experience. 3. **Bug Detection**: Telemetry data can help us detect and fix bugs faster by providing context on how and when issues occur. 4. **User Experience**: Insights from telemetry data guide us in making Encore more intuitive and user-friendly. ## How Data is Collected Encore collects data in a way that prioritizes user privacy and security. Here's how we do it: 1. **User Identifiable Data**: The data collected includes identifiable information that helps us understand specific user interactions and contexts. 2. **Types of Data**: We collect data on usage patterns, performance metrics, and error reports. 3. **Secure Transmission**: All data is transmitted securely using industry-standard encryption protocols. 4. **Minimal Impact**: Data collection is designed to have minimal impact on Encore's performance. ### Example of Data Being Sent Here is an example of the type of data that is sent: ```json { "event": "app.create", "anonymousId": "a-uuid-unique-for-the-installation", "properties": { "error": false, "lang": "go", "template": "graphql" } } ``` ## Data We Don't Collect At Encore, we prioritize your privacy and ensure that no sensitive data is collected through our telemetry. Specifically, we do not collect: 1. **Environment Variables**: We do not collect any environment variables set in your development or production environments. 2. **File Paths**: The specific paths of your files and directories are not collected. 3. **Contents of Files**: We do not access or collect the contents of your code files or any other files in your projects. 4. **Logs**: No log files from your application or development environment are collected. 5. **Serialized Errors**: We do not collect serialized errors that may contain sensitive information. Our goal is to gather useful data that helps improve Encore while ensuring that your sensitive information remains private and secure. ## Disabling Telemetry While telemetry helps us improve Encore, we understand that some users may prefer to opt out. Disabling telemetry is straightforward and can be done in two ways: 1. **Using the CLI Command**: You can disable telemetry by executing a simple command in your terminal. ```sh encore telemetry disable ``` 2. **Setting an Environment Variable**: Alternatively, you can disable telemetry by setting the `DISABLE_ENCORE_TELEMETRY` environment variable. ```sh export DISABLE_ENCORE_TELEMETRY=1 ``` 3. **Confirmation**: After disabling telemetry, either by the CLI command or environment variable, you will receive a confirmation message indicating that telemetry has been successfully disabled. 4. **Re-enabling Telemetry**: If you decide to re-enable telemetry later, you can do so with the following CLI command: ```sh encore telemetry enable ``` ## Debugging Telemetry For users who want more visibility into what telemetry data is being sent, you can enable debug mode: 1. **Setting Debug Mode**: Enable debug mode by setting the `ENCORE_TELEMETRY_DEBUG` environment variable. ```sh export ENCORE_TELEMETRY_DEBUG=1 ``` 2. **Log Statements**: When debug mode is enabled, a log statement prepended by `[telemetry]` will be printed every time telemetry data is sent. ## Conclusion Telemetry is a vital tool for improving Encore, but we respect your choice regarding data sharing. With easy-to-use commands and environment variables, you can manage your telemetry settings as you see fit. If you have any further questions or need assistance, please refer to our support documentation or contact our support team. Thank you for helping us make Encore better! --- Source: https://encore.dev/docs/platform/migration/migrate-to-encore Section: Guides # Migrating an existing system to Encore > Approaches for adopting Encore By building your application with the Encore open-source framework, you unlock powerful features such as the [local development tools](https://encore.dev/docs/dev-dash), [automatic infrastructure provisioning](https://encore.dev/docs/platform/infrastructure/infra), [distributed tracing](https://encore.dev/docs/tracing), and [service catalog](https://encore.dev/docs/service-catalog). **The good news: you don't need a complete rewrite.** This guide shows you how to adopt Encore incrementally, so you can start benefiting immediately while gradually migrating your existing system. If the migration is driven by your current infrastructure or hosting platform, see [Coming from an IaC tool](https://encore.dev/docs/platform/migration/from-iac) or [Coming from a PaaS](https://encore.dev/docs/platform/migration/from-paas) for resource ownership, imports, and platform-service mappings. ## Why incremental migration? Incremental migration is more reliable than a complete rewrite. Here's why: - **Immediate value** - Start benefiting from Encore's features when developing your next new service. - **Lower risk** - Small, controlled changes instead of a single high-stakes big-bang launch. - **Ship faster** - Deliver improvements incrementally rather than waiting for a complete rewrite. ## Choose your migration strategy We recommend two approaches: 1. **Service by service** (Recommended) - Migrate services one at a time. Run Encore alongside your legacy system, integrated via APIs. 2. **Forklift migration** - Move your entire application in one shot using a catch-all handler, then refactor incrementally. ![Migration options](https://encore.dev/assets/docs/migration-diagram.png) ### Need help? We've helped 100+ teams adopt Encore and we're happy to answer your questions and provide advice to help you with your migration. [Email us](mailto:hello@encore.dev) to ask questions, or [book a 1:1 call](https://encore.dev/book) to discuss your specific situation. **Enterprise customers**: the Encore platform can adapt to your unique infrastructure—Kubernetes clusters, VPCs, security policies, and compliance needs—typically within days. [Contact us](https://encore.dev/book) to discuss your requirements. ## Service by service migration (Recommended) Migrate services one at a time while your Encore application runs alongside your legacy system, integrated through APIs. ### Key benefits - **Full Encore features immediately** - Get automatic infrastructure provisioning, distributed tracing, and architecture diagrams for each migrated service. - **Independent services** - Each service is self-contained with no cross-application dependencies. - **Simple integration** - Services communicate via APIs. - **Flexible deployment** - Deploy to your existing Kubernetes cluster, or let the Encore platform set up a new project in your cloud (AWS/GCP). - **Better developer experience** - Start building with modern tooling right away. ### Deployment options Choose how to deploy your Encore application: - **Your Kubernetes cluster** - Deploy directly to your existing Kubernetes infrastructure. Run Encore alongside legacy systems securely in the same environment. - **Encore-managed in your cloud account** - Let Encore handle all infrastructure provisioning and management in your AWS or GCP account, and deploy within your existing VPC and security setup. **Enterprise**: We can adapt to your specific network topology, security policies, and compliance requirements — typically within days. [Contact us](https://encore.dev/book) to discuss your requirements. _Google Cloud example architecture:_ ![Migration options](https://encore.dev/assets/docs/gcp-diagram.png) ### Which services to migrate first? Start small and build confidence: - **Low-risk, high-value** - Validate the approach before tackling complex systems. - **Frequently changed** - Get immediate developer experience benefits where it matters most. - **Clear boundaries** - Services with well-defined APIs are easier to migrate. - **Fewer dependencies** - Less connected to legacy infrastructure means simpler migration. ### Practical steps #### 1. Create an Encore app and integrate with GitHub The first step in any project is to create an Encore app. If you've not tried Encore before, we recommend starting by following the [Quick Start Guide](https://encore.dev/docs/quick-start). Once you've created you app, [integrate it with your GitHub repository](https://encore.dev/docs/platform/integrations/github) and you'll get automatic [Preview Environments](https://encore.dev/docs/platform/deploy/preview-environments) for every Pull Request. #### 2. Build your services and APIs Since Encore is designed to build distributed systems, it should be straightforward to build a new system that integrates with your existing backend through APIs. See the [defining APIs documentation](https://encore.dev/docs/ts/primitives/defining-apis) for more details. Should you want to accept webhooks, that's simple to do using Encore's [Raw endpoints](https://encore.dev/docs/ts/primitives/raw-endpoints). You can also generate API clients in several languages, which makes it simple to integrate with frontends or other systems. See the [Client Generation documentation](https://encore.dev/docs/ts/cli/client-generation) for more details. #### 3. Deploy alongside your existing backend **Deploy to Kubernetes** The Encore platform can deploy directly to your existing Kubernetes cluster: - Run in the same secure environment as your legacy systems - Services communicate within your existing VPC - Gradually shift traffic using your load balancer or service mesh - Use your current cost management and billing - Maintain compliance with your governance policies [Contact us](https://encore.dev/book) to discuss Kubernetes deployment. **Deploy to new Encore-managed infrastructure in your cloud** [Connect your AWS or GCP account](https://encore.dev/docs/platform/deploy/own-cloud) to deploy in your existing environment: - Same VPC as your legacy backend (or a new one) - Your current cost management and billing - Maintain compliance with your governance policies See [infrastructure docs](https://encore.dev/docs/platform/infrastructure/infra#production-infrastructure) for details. #### Integration patterns Your Encore and legacy systems communicate through APIs: - **Legacy → Encore**: Use Encore-generated API clients - **Encore → Legacy**: Use your existing API communication protocol (Encore is not opinionated) - **Authentication**: Choose to deploy an authentication gateway in front of Encore or implement authentication directly in your Encore app - **Events**: Use Encore's built-in Pub/Sub support for loose coupling #### 4. Expand your migration Continue migrating services incrementally. Strategies to consider: - **Related services**: Migrate services that interact frequently to maximize tracing benefits - **High-churn areas**: Move frequently changed services first - **New features**: Build new functionality in Encore from the start - **Critical paths**: Once confident, migrate business-critical services ## Forklift migration using a catch-all handler Should you prefer, you can use a forklift migration strategy to move your entire application to Encore in one step by wrapping your existing HTTP router in a catch-all handler. ### When to consider this approach This strategy works well when: - Your existing system is a monolith or smaller distributed system - The codebase relies primarily on infrastructure primitives supported by Encore (microservices, databases, pub/sub, caching, object storage, cron jobs, and secrets) - You want to quickly consolidate everything in one place - You're prepared to incrementally refactor to unlock full Encore features like tracing and automatic API documentation ### Trade-offs **Benefits:** - **Quick consolidation**: Get everything in one place from the start. - **Immediate access to core features**: Quickly use Encore's CI/CD system, secrets manager, and deployment capabilities. - **Single codebase**: Simplified development and deployment workflow. **Limitations:** - **Limited initial visibility**: Advanced features like [distributed tracing](https://encore.dev/docs/tracing) and [architecture diagrams](https://encore.dev/docs/architecture) require the [Encore application model](https://encore.dev/docs/understanding-encore) and won't work immediately. - **Requires refactoring**: You'll need to incrementally break out endpoints to unlock full Encore capabilities. - **All-at-once risk**: Unlike service-by-service migration, this is a bigger initial change. ### Practical steps Here follows a quick summary of the high-level steps of a forklift migration. Find more in-depth instructions in the full [forklift migration guide](https://encore.dev/docs/ts/migration/express-migration#forklift-migration-quick-start). #### 1. Create an app and structure your code To start, create an Encore application and copy over the code from your existing repository. In order to run your application with Encore, it needs to follow the expected [application structure](https://encore.dev/docs/app-structure), which involves placing the `encore.app` and `package.json` files in the repository root. This should be straightforward to do with minor modifications. As an example, a single service application might look like this on disk: ``` /my-app ├── package.json ├── encore.app ├── // ... other project files │ ├── encore.service.ts // defines your service root ├── api.ts // API endpoints ├── db.ts // Database definition ``` You can also have services nested inside a `backend` folder if you prefer. #### 2. Create a catch-all handler for your HTTP router Now let's mount your existing HTTP router under a [Raw endpoint](https://encore.dev/docs/ts/primitives/raw-endpoints), which is an Encore API endpoint type that gives you access to the underlying HTTP request. Here's a basic code example: **Encore.ts:** ```ts import { api } from "encore.dev/api"; export const migrationHandler = api.raw( { expose: true, method: "*", path: "/api/*path" }, async (req, resp) => { // pass request to existing router } ); ``` **Encore.go:** ```go import "net/http" //encore:api public raw path=/api/*path func MigrationHandler(w http.ResponseWriter, req *http.Request) { // pass request to existing router } ``` By mounting your existing HTTP router in this way, it will work as a catch-all handler for all HTTP requests and responses. This should make your application deployable through Encore with little refactoring. #### 3. Iteratively fix remaining compilation errors Exactly what remains to make your application deployable with Encore will depend on your specific app. As you run your app locally, using `encore run`, Encore will parse and compile it, and give you compilation errors to inform what needs to be adjusted. By iteratively making adjustments, you should relatively quickly be able to get your application up and running with Encore. #### 4. Refactor incrementally to unlock Encore features Once your application is deployed, gradually break out specific endpoints using Encore's [API declarations](https://encore.dev/docs/ts/primitives/defining-apis) and introduce infrastructure declarations using the Encore backend frameworks. This incremental refactoring will: - Enable Encore to understand your application structure - Unlock powerful features like distributed tracing and architecture diagrams - Improve observability and debugging capabilities - Make your codebase more maintainable and easier to evolve Start with the most frequently modified endpoints or the most critical user flows to maximize the value of refactoring efforts. ## Conclusion Incremental migration lets you adopt Encore without the risk of a complete rewrite. **Service by service migration** is the recommended approach—it gives you Encore's full feature set immediately while running safely alongside your existing systems. **Enterprise customers** benefit from flexible deployment options, including Kubernetes integration and customization that typically takes just days to set up. ### Have questions? We've helped 100+ teams adopt Encore and we're happy to answer your questions and provide advice to help you with your migration. - [Book a call](https://encore.dev/book) to get 1:1 assistance - [Email us](mailto:hello@encore.dev) to ask questions - [Join Discord](https://encore.dev/discord) to discuss with other developers using Encore --- Source: https://encore.dev/docs/platform/migration/from-iac Section: Guides # Coming from an IaC tool > Map existing infrastructure workflows to Encore Encore is a backend development platform. You define APIs and services with its TypeScript or Go SDK and declare the infrastructure they use in the same application code. Encore derives an [application model](https://encore.dev/docs/understanding-encore) from those declarations and uses it throughout development and deployment. When you deploy with the Encore platform, Encore provisions supported resources in AWS or GCP. For self-hosted environments, your existing infrastructure tooling continues to provision them. Supported resources include SQL databases, Pub/Sub, object storage, caches, cron jobs, and secrets. They are declared in application code using [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go). You do not need to replace your existing infrastructure configuration at once. Encore can manage supported application resources while your current IaC tool continues to manage shared, organization-level, or unsupported infrastructure. ## Resource declarations An Encore declaration identifies the logical resource the application uses without selecting its cloud implementation. For example, this declaration defines an `orders` database and its migration directory: **Encore.ts:** ```ts import { SQLDatabase } from "encore.dev/storage/sqldb"; const db = new SQLDatabase("orders", { migrations: "./migrations" }); ``` **Encore.go:** ```go import "encore.dev/storage/sqldb" var db = sqldb.NewDatabase("orders", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` The logical resource is the named database used by the application. Its physical implementation may be local Postgres, a database provisioned by the Encore platform, or a supported existing database connected to the environment. With the Encore platform, deployment properties are configured separately for each environment. When self-hosting, an [infra config file](https://encore.dev/docs/self-host/configure-infra) maps each logical resource to infrastructure provisioned by your existing tooling. ## Choose the deployment model First decide whether the environment will use the Encore platform or be self-hosted. With the Encore platform, choose one of the following paths for each supported resource: | Resource path | Approach | |---|---| | **Managed by Encore platform** | Declare the resource in application code and configure its deployment properties per environment. | | **Already provisioned** | Follow the resource-specific import guide to connect an Encore declaration to the physical resource and determine which settings Encore manages afterward. | | **Managed by your IaC tool** | Keep shared and unsupported infrastructure in the existing configuration and connect to it as an external dependency. | Encore-managed resources can coexist with imported and external infrastructure. The ownership boundary can also change as services are migrated. Importing connects a logical declaration to an existing physical resource. Do not continue applying changes from the IaC tool unless the resource's import guide says that ownership remains there. For self-hosted environments, your IaC tool provisions all physical infrastructure. The infra config file maps supported Encore declarations to those resources. ## How the concepts map | IaC concept | Encore equivalent | |---|---| | Resource declaration | A typed resource declaration in [TypeScript](https://encore.dev/docs/ts) or a package-level declaration in [Go](https://encore.dev/docs/go) | | Provider configuration | The cloud provider and infrastructure settings selected for an Encore Cloud environment | | Infrastructure and deployment variables | Per-environment [infrastructure configuration](https://encore.dev/docs/platform/infrastructure/configuration) | | Secret variables | Encore secrets for [TypeScript](https://encore.dev/docs/ts/primitives/secrets) or [Go](https://encore.dev/docs/go/primitives/secrets) | | Other application configuration | Application code or an external configuration service; Encore has no general-purpose non-secret configuration primitive | | Explicit resource dependencies | Relationships derived from resource usage and service calls in the [application model](https://encore.dev/docs/understanding-encore) | | Modules | No direct equivalent; Encore services organize application code and the resources it uses | | `count`, `for_each`, or generated resources | No direct equivalent for Encore resources; declarations must be statically discoverable | | State file | The application model is derived from source; Encore platform keeps a separate record of the physical infrastructure managed for each environment | | Plan | No direct equivalent; builds validate the application model, and Encore platform provides infrastructure change workflows | | Apply | Encore platform applies managed infrastructure changes during deployment; for self-hosted environments, the existing IaC workflow provisions infrastructure | ## Use existing infrastructure The Encore platform can connect to selected existing resources instead of recreating them. Supported imports include: - [AWS RDS](https://encore.dev/docs/platform/infrastructure/aws/import-rds) and [Google Cloud SQL](https://encore.dev/docs/platform/infrastructure/gcp/import-cloud-sql) instances - [Amazon S3](https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket) and [Google Cloud Storage](https://encore.dev/docs/platform/infrastructure/gcp/import-gcs-bucket) buckets - [Amazon SNS](https://encore.dev/docs/platform/infrastructure/aws/import-sns-topic) and [Google Cloud Pub/Sub](https://encore.dev/docs/platform/infrastructure/gcp/import-pubsub-topic) topics At the environment level, the Encore platform can also deploy into an existing [GKE cluster](https://encore.dev/docs/platform/infrastructure/import-kubernetes-cluster) or [Google Cloud project](https://encore.dev/docs/platform/infrastructure/gcp/import-project). Terraform can also reference infrastructure provisioned by Encore. The [Encore Terraform Provider](https://encore.dev/docs/platform/integrations/terraform) exposes Encore resources as read-only data sources, allowing an existing Terraform configuration to reference an Encore database, cache, or Pub/Sub topic. ## What usually remains in IaC Existing infrastructure configuration commonly continues to manage: - organization-level networking and account configuration - DNS and third-party providers - infrastructure without an Encore primitive - resources shared with systems outside the Encore application - resources whose number or identity is determined at runtime **Encore.ts:** Connect these resources to the application as external dependencies and store sensitive connection details in secrets for [TypeScript](https://encore.dev/docs/ts/primitives/secrets) or [Go](https://encore.dev/docs/go/primitives/secrets). **Encore.go:** Connect these resources to the application as external dependencies and store sensitive connection details in secrets for [TypeScript](https://encore.dev/docs/go/primitives/secrets) or [Go](https://encore.dev/docs/go/primitives/secrets). An external dependency remains outside Encore's provisioning lifecycle. Application code connects to it using its normal client or SDK, while your existing tooling continues to manage the resource. ## Plan an incremental migration Start with one service and list the infrastructure it uses. Then: 1. Inventory the resources used by the application and record which systems depend on them. 2. Map supported application resources to Encore primitives. 3. Decide whether the environment will use the Encore platform or be self-hosted. 4. For Encore Cloud environments, decide which resources Encore should provision, connect to as existing resources, or leave under the current IaC workflow. 5. Run and test the migrated service locally with `encore run` and `encore test`. 6. For the Encore platform, create an environment and follow the relevant import workflow for existing resources. For self-hosting, provision the physical resources and create the infra config mapping. 7. Deploy the service and verify its resource connections before migrating another service. See [Development workflow](https://encore.dev/docs/platform/workflow) for local, preview, and production environments. [Understanding Encore](https://encore.dev/docs/understanding-encore) explains how Encore discovers resources and their relationships. For strategies to migrate an existing backend one service at a time or through a catch-all handler, see [Migrating an existing system to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore). ## Tool-specific guides - [Terraform](https://encore.dev/docs/platform/migration/from-terraform): map resources, modules, state, imports, and plan/apply workflows. - [Pulumi](https://encore.dev/docs/platform/migration/from-pulumi): map projects, stacks, configuration, outputs, and Pulumi state. - [OpenTofu](https://encore.dev/docs/platform/migration/from-opentofu): plan ownership boundaries, imports, and coexistence with existing OpenTofu configuration. - [AWS CDK](https://encore.dev/docs/platform/migration/from-cdk): map constructs, stacks, CloudFormation resources, and CDK deployment workflows. --- Source: https://encore.dev/docs/platform/migration/from-terraform Section: Guides # Coming from Terraform > How Terraform concepts map to Encore Terraform configuration describes the desired infrastructure and its provider-specific settings. In Encore, the TypeScript or Go SDK declarations describe the APIs, services, and infrastructure used by the backend. They form an [application model](https://encore.dev/docs/understanding-encore), while deployment-specific properties are configured separately for each environment. For migration paths that also apply to Pulumi, OpenTofu, and AWS CDK, see [Coming from an IaC tool](https://encore.dev/docs/platform/migration/from-iac). ## How it works The following examples declare a Postgres database in Terraform and Encore. ```hcl resource "aws_db_instance" "site" { identifier = "site" engine = "postgres" instance_class = "db.t4g.medium" allocated_storage = 20 storage_type = "gp3" db_name = "site" username = var.db_username password = var.db_password db_subnet_group_name = aws_db_subnet_group.main.name vpc_security_group_ids = [aws_security_group.db.id] backup_retention_period = 7 } ``` **Encore.ts:** ```ts import { SQLDatabase } from "encore.dev/storage/sqldb"; const siteDB = new SQLDatabase("site", { migrations: "./migrations", }); ``` **Encore.go:** ```go import "encore.dev/storage/sqldb" var siteDB = sqldb.NewDatabase("site", sqldb.DatabaseConfig{ Migrations: "./migrations", }) ``` The Terraform example defines the AWS implementation and its deployment properties. The Encore declaration defines the logical database name and its migration directory. The Encore platform configures the managed database separately for each environment through [infrastructure configuration](https://encore.dev/docs/platform/infrastructure/configuration). ## Where each setting is configured Configuration is divided across three locations: | Place | Holds | How you set it | |---|---|---| | **Code** | Logical resources and application-level properties | Arguments to resource constructors in the application | | **Encore platform configuration** | Deployment properties for each environment | The [Encore platform dashboard](https://app.encore.dev) and, where supported, the cloud provider console | | **Infra config** | Resource mappings and runtime connectivity for self-hosted deployments | An [`infra.config.json`](https://encore.dev/docs/self-host/configure-infra) file passed to `encore build docker --config` | The infra config file applies to self-hosted deployments. The Encore platform manages the corresponding configuration for environments deployed through the platform. For a SQL database: | Setting | Place | |---|---| | Database name, migrations directory | Code | | Cloud provider, database engine, and capacity | Encore platform configuration | | Storage, backups, and networking | Encore platform configuration or cloud provider console | | Host, credentials, and TLS | Infra config for self-hosted deployments | ## Converting Terraform resources ### Databases ```hcl resource "aws_db_instance" "site" { ... } ``` **Encore.ts:** ```ts import { SQLDatabase } from "encore.dev/storage/sqldb"; const SiteDB = new SQLDatabase("site", { migrations: "./migrations" }); ``` **Encore.go:** ```go import "encore.dev/storage/sqldb" var SiteDB = sqldb.NewDatabase("site", sqldb.DatabaseConfig{Migrations: "./migrations"}) ``` **Encore.ts:** Schema changes go in `./migrations` as numbered SQL files, applied on deploy. See [Databases](https://encore.dev/docs/ts/primitives/databases). **Encore.go:** Schema changes go in `./migrations` as numbered SQL files, applied on deploy. See [Databases](https://encore.dev/docs/go/primitives/databases). ### Pub/Sub topics and subscriptions A topic, its queue, the subscription, and the queue policy granting the topic write access: ```hcl resource "aws_sns_topic" "site_added" { name = "site-added" } resource "aws_sqs_queue" "check_site" { name = "check-site" } resource "aws_sns_topic_subscription" "check_site" { topic_arn = aws_sns_topic.site_added.arn protocol = "sqs" endpoint = aws_sqs_queue.check_site.arn } resource "aws_sqs_queue_policy" "check_site" { ... } ``` **Encore.ts:** ```ts import { Topic, Subscription } from "encore.dev/pubsub"; export const SiteAdded = new Topic("site-added", { deliveryGuarantee: "at-least-once", }); const _ = new Subscription(SiteAdded, "check-site", { handler: doCheck }); ``` **Encore.go:** ```go import "encore.dev/pubsub" var SiteAdded = pubsub.NewTopic[*Site]("site-added", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) var _ = pubsub.NewSubscription(SiteAdded, "check-site", pubsub.SubscriptionConfig[*Site]{Handler: DoCheck}, ) ``` **Encore.ts:** Encore derives the queue, the subscription wiring and the IAM policy from those two declarations. See [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub). **Encore.go:** Encore derives the queue, the subscription wiring and the IAM policy from those two declarations. See [Pub/Sub](https://encore.dev/docs/go/primitives/pubsub). ### Object storage ```hcl resource "aws_s3_bucket" "profile_pictures" { bucket = "profile-pictures" } resource "aws_s3_bucket_versioning" "profile_pictures" { ... } ``` **Encore.ts:** ```ts import { Bucket } from "encore.dev/storage/objects"; export const profilePictures = new Bucket("profile-pictures", { versioned: false, }); ``` **Encore.go:** ```go import "encore.dev/storage/objects" var ProfilePictures = objects.NewBucket("profile-pictures", objects.BucketConfig{ Versioned: false, }) ``` **Encore.ts:** See [Object storage](https://encore.dev/docs/ts/primitives/object-storage). **Encore.go:** See [Object storage](https://encore.dev/docs/go/primitives/object-storage). ### Scheduled tasks ```hcl resource "aws_cloudwatch_event_rule" "welcome_email" { schedule_expression = "rate(2 hours)" } resource "aws_cloudwatch_event_target" "welcome_email" { ... } resource "aws_lambda_permission" "welcome_email" { ... } ``` **Encore.ts:** ```ts import { CronJob } from "encore.dev/cron"; const _ = new CronJob("welcome-email", { title: "Send welcome emails", every: "2h", endpoint: sendWelcomeEmail, }); ``` **Encore.go:** ```go import "encore.dev/cron" var _ = cron.NewJob("welcome-email", cron.JobConfig{ Title: "Send welcome emails", Every: 2 * cron.Hour, Endpoint: SendWelcomeEmail, }) ``` **Encore.ts:** The target is a reference to the endpoint function rather than an ARN. See [Cron jobs](https://encore.dev/docs/ts/primitives/cron-jobs). **Encore.go:** The target is a reference to the endpoint function rather than an ARN. See [Cron jobs](https://encore.dev/docs/go/primitives/cron-jobs). ### Secrets ```hcl resource "aws_secretsmanager_secret" "github_token" { name = "GitHubAPIToken" } ``` **Encore.ts:** ```ts import { secret } from "encore.dev/config"; const githubToken = secret("GitHubAPIToken"); ``` **Encore.go:** ```go var secrets struct { GitHubAPIToken string } ``` **Encore.ts:** Values are set per environment with `encore secret set`. See [Secrets](https://encore.dev/docs/ts/primitives/secrets). **Encore.go:** Values are set per environment with `encore secret set`. See [Secrets](https://encore.dev/docs/go/primitives/secrets). ## Concept mapping **Declaring infrastructure** | Terraform | Encore | |---|---| | `resource` block | Resource constructor in application code | | `provider` block | Cloud provider is chosen per environment in the dashboard | | Resource reference (`aws_sns_topic.x.arn`) | Language-level import of the resource object | | `depends_on` | Usually derived from resource usage and service relationships; no direct equivalent | | Modules | No direct equivalent; services organize application code and resources | | `count` and `for_each` | No equivalent for Encore resources; declarations must be statically discoverable | **Configuration and state** | Terraform | Encore | |---|---| | Input variables and `locals` | No single equivalent; infrastructure settings, secrets, and other application configuration are handled separately | | `output` and `var` plumbing | Not required; the consumer imports the object | | `tfvars` per environment | No single equivalent; Encore platform settings, secrets, and other application configuration are handled separately | | Workspaces | Environments are the closest operational concept, but their lifecycle and configuration differ | | State file | The application model is derived from source; Encore platform separately maintains infrastructure state | | Remote state backend | No direct equivalent for platform-managed environments; Encore platform maintains its infrastructure records | **Running it** | Terraform | Encore | |---|---| | `terraform plan` | No direct equivalent; builds validate the application model and Encore platform provides infrastructure change workflows | | `terraform apply` | No direct equivalent; Encore platform deployment provisions supported changes, while self-hosted infrastructure remains in your IaC workflow | | `terraform import` | No direct equivalent; use the resource-specific Encore import workflow to connect supported existing infrastructure | | `data` source for an Encore resource | [Encore Terraform Provider](https://encore.dev/docs/platform/integrations/terraform) data sources | | External migration step | `migrations/` directory, applied on deploy | | Hand-written IAM policies | Derived from supported resource usage in the application model | ## Using Encore with Terraform Encore and Terraform can be used together. Encore can manage supported application resources while Terraform manages other infrastructure, such as DNS, organization-level networking, and third-party providers. ### Pointing Encore at resources you already have Encore provides resource-specific workflows for connecting existing [RDS](https://encore.dev/docs/platform/infrastructure/aws/import-rds), [Cloud SQL](https://encore.dev/docs/platform/infrastructure/gcp/import-cloud-sql), [S3](https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket), [GCS](https://encore.dev/docs/platform/infrastructure/gcp/import-gcs-bucket), [SNS](https://encore.dev/docs/platform/infrastructure/aws/import-sns-topic), and [Pub/Sub](https://encore.dev/docs/platform/infrastructure/gcp/import-pubsub-topic) resources. At the environment level, it can also deploy into an existing [GKE cluster](https://encore.dev/docs/platform/infrastructure/import-kubernetes-cluster) or [GCP project](https://encore.dev/docs/platform/infrastructure/gcp/import-project). These workflows do not read or convert Terraform state. Follow the guide for the resource before changing its Terraform ownership. If ownership moves to Encore, remove the resource from Terraform without destroying it. Terraform recommends replacing its `resource` block with a [`removed` block](https://developer.hashicorp.com/terraform/language/state/remove), then reviewing and applying the plan. `destroy = false` goes inside a nested `lifecycle` block: ```hcl removed { from = aws_s3_bucket.uploads lifecycle { destroy = false } } ``` ### Reading Encore resources from Terraform The [Encore Terraform Provider](https://encore.dev/docs/platform/integrations/terraform) exposes data sources such as `encore_database`, `encore_cache`, and `encore_pubsub_topic`, so Terraform can reference infrastructure Encore created: ```hcl data "encore_pubsub_topic" "topic" { name = "my-topic" env = "my-env" } resource "aws_iot_topic_rule" "rule" { name = "my-rule" sql = "SELECT * FROM 'my-topic'" sns { message_format = "RAW" role_arn = aws_iam_role.role.arn target_arn = data.encore_pubsub_topic.topic.aws_sns.arn } } ``` ### Self-hosting and binding manually Build a Docker image with `encore build docker` and pass an `infra.config.json` that maps each resource in the derived model to a physical resource you provision separately: ```bash encore build docker myapp:latest --config ./infra.config.json ``` Running the build without `--config` prints the resources the file has to account for. See [Configure infrastructure](https://encore.dev/docs/self-host/configure-infra). ## Workflow differences Encore validates statically discoverable resource declarations during the build and configures their deployment per environment. There is no direct equivalent to `terraform plan`, `count`, or `for_each`. Keep unsupported, shared, and dynamically generated infrastructure in Terraform and migrate one ownership boundary at a time. ## Why configuration is split this way Application-level properties stay with the resource declaration. For a SQL database, these include its logical name and migration directory. Deployment properties such as the database engine and capacity are configured per environment. Resource declarations can be used locally and in AWS or GCP environments. New environments select their own infrastructure settings, including [preview environments](https://encore.dev/docs/platform/deploy/preview-environments). Capacity and other deployment settings can be managed through [infrastructure configuration](https://encore.dev/docs/platform/infrastructure/configuration) without changing the resource declaration. The static declarations also let Encore determine which resources exist and which services use them. Encore uses that information to derive IAM policies and validate supported resource usage. For agentic workflows, [AI infrastructure provisioning](https://encore.dev/docs/platform/ai-integration) describes the available controls, and [infrastructure namespaces](https://encore.dev/docs/infra-namespaces) provide isolated local state. See [Understanding Encore](https://encore.dev/docs/understanding-encore) for details on how declarations are analyzed. --- Source: https://encore.dev/docs/platform/migration/from-pulumi Section: Guides # Coming from Pulumi > Map Pulumi programs to Encore resources and environments Pulumi programs execute general-purpose code to construct an infrastructure graph. Encore's TypeScript or Go SDK declarations instead describe the APIs, services, and infrastructure used by the backend. They form an [application model](https://encore.dev/docs/understanding-encore) that Encore uses for local development and deployment, while Pulumi can continue to manage infrastructure outside that model. For the deployment models and migration paths shared by all infrastructure-as-code tools, see [Coming from an IaC tool](https://encore.dev/docs/platform/migration/from-iac). ## How it works The following TypeScript examples define an object storage bucket in Pulumi and Encore: ```ts import * as aws from "@pulumi/aws"; const uploads = new aws.s3.Bucket("uploads"); ``` **Encore.ts:** ```ts import { Bucket } from "encore.dev/storage/objects"; export const uploads = new Bucket("uploads", { versioned: false }); ``` **Encore.go:** ```go import "encore.dev/storage/objects" var Uploads = objects.NewBucket("uploads", objects.BucketConfig{Versioned: false}) ``` The Pulumi resource represents an S3 bucket managed in a Pulumi stack. The Encore declaration identifies the bucket used by the application. Locally, Encore provides development infrastructure. In a cloud environment, Encore can [provision the bucket](https://encore.dev/docs/platform/infrastructure/configuration) or connect the declaration to a [supported existing bucket](https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket). Resource names must be string literals, and declarations must be statically discoverable. This differs from a Pulumi program, which can use normal language control flow to construct an infrastructure graph when the program runs. ## How the concepts map | Pulumi | Encore | |---|---| | Project | Encore application | | Stack | An environment is the closest operational concept, but the lifecycle and configuration differ | | Custom resource | A primitive when Encore supports the resource type; otherwise it remains in Pulumi | | Component resource | No direct equivalent; Encore services organize application code and its resource usage | | Resource inputs | Application-level properties in the declaration, or deployment settings configured per environment | | Resource outputs | Usually not passed through the application; code imports the logical resource object | | Passing a resource `Output` as an input | Resource usage and service calls recorded in the application model; no direct one-to-one equivalent | | `dependsOn` | Usually derived from application relationships; no direct equivalent | | Stack configuration | No single equivalent; infrastructure settings, secrets, and other application configuration are handled separately | | Stack outputs and stack references | No general equivalent; keep Pulumi outputs for infrastructure that remains in Pulumi | | Pulumi state backend | The application model is rebuilt from source; Encore platform separately records the infrastructure it manages | | `pulumi preview` | No direct equivalent; builds validate declarations and Encore platform provides infrastructure change workflows | | `pulumi up` | No direct equivalent; Encore platform deployment provisions supported changes, while self-hosted infrastructure remains in the Pulumi workflow | Pulumi configuration often combines application values with physical infrastructure settings. In Encore, place each value according to its purpose: **Encore.ts:** - Keep logical resource properties, such as a database name and migrations directory, in application code. - Configure capacity, networking, backups, and other deployment properties per [an Encore Cloud environment](https://encore.dev/docs/platform/infrastructure/configuration). - Store sensitive application values in Encore [TypeScript secrets](https://encore.dev/docs/ts/primitives/secrets) or [Go secrets](https://encore.dev/docs/go/primitives/secrets). - Keep configuration for Pulumi-managed resources in the Pulumi stack. **Encore.go:** - Keep logical resource properties, such as a database name and migrations directory, in application code. - Configure capacity, networking, backups, and other deployment properties per [an Encore Cloud environment](https://encore.dev/docs/platform/infrastructure/configuration). - Store sensitive application values in Encore [TypeScript secrets](https://encore.dev/docs/go/primitives/secrets) or [Go secrets](https://encore.dev/docs/go/primitives/secrets). - Keep configuration for Pulumi-managed resources in the Pulumi stack. Encore does not provide a general-purpose non-secret configuration primitive. Ordinary application settings can remain in code or come from an external configuration service. ## Decide what Pulumi continues to manage Pulumi and Encore can run alongside each other. A common boundary is to use Encore for supported resources owned by the application, while Pulumi continues to manage: - account and organization configuration - shared networking and DNS - third-party providers - AWS, Azure, or Kubernetes resources without an Encore primitive - infrastructure shared with applications outside Encore - resources generated dynamically by the infrastructure program The Encore platform provisions on AWS and GCP. Pulumi can continue to manage infrastructure on other providers. For a self-hosted Encore deployment, Pulumi provisions all physical infrastructure and an infra config file maps Encore's logical resources to it. See [Configure infrastructure](https://encore.dev/docs/self-host/configure-infra). ## Migrate existing resources Do not remove a resource from Pulumi state until you have chosen its new owner and reviewed the cutover process. For an Encore Cloud environment, supported existing resources can be connected to an Encore declaration. These include: - [AWS RDS](https://encore.dev/docs/platform/infrastructure/aws/import-rds) and [Google Cloud SQL](https://encore.dev/docs/platform/infrastructure/gcp/import-cloud-sql) - [Amazon S3](https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket) and [Google Cloud Storage](https://encore.dev/docs/platform/infrastructure/gcp/import-gcs-bucket) - [Amazon SNS](https://encore.dev/docs/platform/infrastructure/aws/import-sns-topic) and [Google Cloud Pub/Sub](https://encore.dev/docs/platform/infrastructure/gcp/import-pubsub-topic) Connecting a resource to Encore is separate from Pulumi's resource import feature. Pulumi import adds an existing cloud resource to a Pulumi stack and state. Encore's import workflow connects a logical application resource to existing physical infrastructure. Follow the Encore guide for that resource to determine what Encore manages afterward. If Pulumi should retain ownership, leave the resource in its Pulumi stack. The Encore application can use it as an external dependency, with sensitive connection details stored in Encore secrets. When moving ownership away from Pulumi, set Pulumi's [`retainOnDelete`](https://www.pulumi.com/docs/iac/concepts/resources/options/retainondelete/) resource option and run `pulumi up` before removing the declaration. Then preview and apply the removal so Pulumi stops managing the resource without deleting it. Verify the resource in the target Encore environment before completing the handoff. If you use a different state workflow, confirm that it also retains the physical resource. ## Migrate incrementally Start with one service rather than an entire Pulumi project: 1. List the cloud resources used by the service and identify their Pulumi stacks. 2. Map databases, Pub/Sub, object storage, caches, cron jobs, and secrets to [Encore primitives](https://encore.dev/docs/ts). 3. Mark shared, unsupported, and dynamically generated infrastructure to remain in Pulumi. 4. Choose the Encore platform or self-hosting for the target environment. 5. Add the Encore declarations and test the service with `encore run` and `encore test`. 6. For the Encore platform, provision new resources or follow the resource-specific import guide. For self-hosting, map the declarations to Pulumi-provisioned infrastructure. 7. Deploy and verify connectivity, data, permissions, and rollback procedures. 8. Change Pulumi ownership only after the Encore environment is verified. Repeat the process for the next service. Stack references and exported outputs may reveal dependencies that must remain available during the migration. ## Workflow differences Encore validates statically discoverable resource declarations during the build and configures their deployment per environment. It does not replace Pulumi previews or stack state. Keep unsupported, shared, and dynamically generated infrastructure in Pulumi, and migrate one ownership boundary at a time. For service-by-service application migration patterns, see [Migrating an existing system to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore). --- Source: https://encore.dev/docs/platform/migration/from-opentofu Section: Guides # Coming from OpenTofu > Map OpenTofu configuration to Encore resources and environments OpenTofu configuration describes desired infrastructure through provider resources. Encore's TypeScript or Go SDK declarations instead describe the APIs, services, and infrastructure used by the backend. They form an [application model](https://encore.dev/docs/understanding-encore) that Encore uses for local development and deployment. OpenTofu can continue to manage infrastructure outside that model or provision all physical resources for a self-hosted environment. OpenTofu aims to maintain compatibility with Terraform configurations, so many of the mappings in [Coming from Terraform](https://encore.dev/docs/platform/migration/from-terraform) also apply. See [Coming from an IaC tool](https://encore.dev/docs/platform/migration/from-iac) for the deployment models shared by these tools. This guide focuses on OpenTofu state, ownership, and running both systems together. ## How it works The following examples define an AWS S3 bucket in OpenTofu and an object storage bucket in Encore: ```hcl resource "aws_s3_bucket" "uploads" { bucket = "uploads" } ``` **Encore.ts:** ```ts import { Bucket } from "encore.dev/storage/objects"; export const uploads = new Bucket("uploads", { versioned: false }); ``` **Encore.go:** ```go import "encore.dev/storage/objects" var Uploads = objects.NewBucket("uploads", objects.BucketConfig{Versioned: false}) ``` The OpenTofu resource selects an AWS implementation. The Encore declaration identifies the logical bucket used by the application. For each environment, Encore can provide local development infrastructure, [provision the bucket](https://encore.dev/docs/platform/infrastructure/configuration), or connect the declaration to a [supported existing bucket](https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket). This is not a source-to-source conversion. OpenTofu can describe resources from a broad provider ecosystem. Encore primitives cover application resources such as databases, Pub/Sub, object storage, caches, cron jobs, and secrets. ## How the concepts map | OpenTofu | Encore | |---|---| | Root module | No direct equivalent; an Encore application contains services and resource declarations | | Child module | No direct equivalent; services organize application code and its resources | | Provider configuration | Cloud provider and deployment settings selected for an Encore Cloud environment | | `resource` block | An Encore primitive when the resource type is supported | | Resource reference | Import and use the resource object in application code | | `data` source | No general equivalent; query external systems using their client or keep the lookup in OpenTofu | | Variables and locals | No single equivalent; infrastructure settings, secrets, and other application configuration are handled separately | | Outputs and remote state | No general equivalent; retain outputs for infrastructure that remains in OpenTofu | | `depends_on` | Usually derived from application resource usage and service calls; no direct equivalent | | `count` and `for_each` | No equivalent for Encore resources; declarations must be statically discoverable | | Workspace | An Encore environment is the closest operational concept, but the lifecycle and configuration differ | | Backend and state | The application model is rebuilt from source; Encore platform separately records managed infrastructure | | `tofu plan` | No direct equivalent; builds validate declarations and Encore platform provides infrastructure change workflows | | `tofu apply` | No direct equivalent; Encore platform deployment provisions supported changes, while self-hosted infrastructure remains in the OpenTofu workflow | For the Encore platform, configure capacity, networking, backups, and related deployment properties per [environment](https://encore.dev/docs/platform/infrastructure/configuration). For self-hosting, OpenTofu provisions the physical resources and an infra config file maps each Encore declaration to its runtime connection details. ## Run Encore with OpenTofu You can keep OpenTofu for infrastructure outside Encore's application model. This commonly includes: - organization and account configuration - shared networking and DNS - third-party providers - infrastructure without an Encore primitive - resources shared with non-Encore systems - resources created with `count`, `for_each`, or other dynamic configuration **Encore.ts:** Application code can connect to those resources through their normal SDKs. Store credentials and other sensitive values in Encore [TypeScript secrets](https://encore.dev/docs/ts/primitives/secrets) or [Go secrets](https://encore.dev/docs/go/primitives/secrets). **Encore.go:** Application code can connect to those resources through their normal SDKs. Store credentials and other sensitive values in Encore [TypeScript secrets](https://encore.dev/docs/go/primitives/secrets) or [Go secrets](https://encore.dev/docs/go/primitives/secrets). The Encore platform environments can combine [resources provisioned by Encore](https://encore.dev/docs/platform/infrastructure/configuration) with supported existing resources. At the environment level, Encore can also deploy into an existing [GKE cluster](https://encore.dev/docs/platform/infrastructure/import-kubernetes-cluster) or [Google Cloud project](https://encore.dev/docs/platform/infrastructure/gcp/import-project). For self-hosted environments, keep all physical resource ownership in OpenTofu. See [Configure infrastructure](https://encore.dev/docs/self-host/configure-infra). ## State and existing resources OpenTofu state maps resource addresses in configuration to physical objects. Encore does not consume an OpenTofu state file. Its application model is derived from source, and the Encore platform maintains separate records for infrastructure managed through the platform. Choose one provisioning owner for each physical resource. Continuing to apply an OpenTofu resource after Encore has taken ownership can produce conflicting changes. The Encore platform has resource-specific workflows for connecting existing infrastructure: - [AWS RDS](https://encore.dev/docs/platform/infrastructure/aws/import-rds) and [Google Cloud SQL](https://encore.dev/docs/platform/infrastructure/gcp/import-cloud-sql) - [Amazon S3](https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket) and [Google Cloud Storage](https://encore.dev/docs/platform/infrastructure/gcp/import-gcs-bucket) - [Amazon SNS](https://encore.dev/docs/platform/infrastructure/aws/import-sns-topic) and [Google Cloud Pub/Sub](https://encore.dev/docs/platform/infrastructure/gcp/import-pubsub-topic) OpenTofu import and Encore import serve different purposes. An OpenTofu `import` block or `tofu import` operation brings an existing object into OpenTofu state. An Encore import connects a logical application declaration to an existing physical resource. Review the Encore resource guide before changing OpenTofu state or configuration. If OpenTofu retains ownership, leave the resource in state and treat it as an external dependency. If ownership moves to Encore, verify the new connection and then replace the resource with a [`removed` block](https://opentofu.org/docs/language/resources/syntax/#removing-resources), with `destroy = false` inside a nested `lifecycle` block: ```hcl removed { from = aws_s3_bucket.uploads lifecycle { destroy = false } } ``` Review the plan before applying it so OpenTofu forgets the resource without destroying the physical object. ## Migrate incrementally 1. Inventory the resources used by one service, including module and remote-state dependencies. 2. Map supported application resources to [TypeScript primitives](https://encore.dev/docs/ts) or [Go primitives](https://encore.dev/docs/go). 3. Leave shared, unsupported, and dynamically generated infrastructure in OpenTofu. 4. Decide whether each target environment will use the Encore platform or be self-hosted. 5. Add resource declarations and test locally with `encore run` and `encore test`. 6. For the Encore platform, provision new resources or follow the relevant import workflow. For self-hosting, retain OpenTofu provisioning and create the infra config mapping. 7. Deploy the service and verify resource connectivity and data before changing OpenTofu state. 8. Repeat for the next service. Keep existing OpenTofu outputs available until every consumer has moved. If separate root modules read each other's state, record that dependency before changing any resource owner. ## Workflow differences Encore validates statically discoverable resource declarations during the build and configures their deployment per environment. It does not replace OpenTofu plans, modules, or state. Keep unsupported, shared, and dynamically generated infrastructure in OpenTofu, and migrate one ownership boundary at a time. See [Development workflow](https://encore.dev/docs/platform/workflow) for the path from local development to preview and production environments. For service-by-service migration patterns, see [Migrating an existing system to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore). --- Source: https://encore.dev/docs/platform/migration/from-cdk Section: Guides # Coming from AWS CDK > Map CDK constructs to Encore resources and environments AWS CDK applications use constructs to define AWS infrastructure and synthesize it into CloudFormation templates. Encore's TypeScript or Go SDK declarations instead describe the APIs, services, and infrastructure used by the backend. They form an [application model](https://encore.dev/docs/understanding-encore) that Encore uses for local development and deployment, while CDK can continue to manage AWS infrastructure outside that model. For migration choices that apply to all infrastructure-as-code tools, see [Coming from an IaC tool](https://encore.dev/docs/platform/migration/from-iac). ## How it works The following TypeScript examples define an object storage bucket in AWS CDK and Encore: ```ts import * as s3 from "aws-cdk-lib/aws-s3"; const uploads = new s3.Bucket(this, "Uploads", { versioned: false, }); ``` **Encore.ts:** ```ts import { Bucket } from "encore.dev/storage/objects"; export const uploads = new Bucket("uploads", { versioned: false }); ``` **Encore.go:** ```go import "encore.dev/storage/objects" var Uploads = objects.NewBucket("uploads", objects.BucketConfig{Versioned: false}) ``` The CDK construct contributes one or more AWS resources to a CloudFormation stack. The Encore declaration identifies a logical bucket used by the application. In a cloud environment, Encore can [provision the bucket](https://encore.dev/docs/platform/infrastructure/configuration) or connect the declaration to a [supported existing bucket](https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket). During local development, Encore provides local development infrastructure. Encore declarations are not CDK constructs. CDK construct code runs during synthesis and can generate an arbitrary CloudFormation graph. Encore uses static analysis, so primitive declarations must use literal names and appear where the parser can discover them. ## How the concepts map | AWS CDK | Encore | |---|---| | App | No direct equivalent; an Encore application is derived from services, APIs, and resources in source | | Stack | An environment is the closest deployment concept, but it is not a CloudFormation stack | | L1, L2, or L3 construct | An Encore primitive when the application resource is supported; otherwise keep the construct in CDK | | Construct tree | The application model records services, APIs, resources, and their usage relationships | | Construct ID and logical ID | No direct equivalent; an Encore resource name identifies the logical application resource | | Tokens and references | Application code imports and uses the logical resource object | | Explicit dependencies | Usually derived from resource usage and service calls; no direct equivalent | | Context and stack parameters | No single equivalent; infrastructure settings, secrets, and other application configuration are handled separately | | CloudFormation output or cross-stack reference | No general equivalent; retain outputs for CDK-managed infrastructure | | `cdk synth` | No direct equivalent; Encore builds derive and validate the application model | | `cdk diff` | No direct equivalent; Encore platform provides infrastructure change workflows | | `cdk deploy` | No direct equivalent; Encore platform deployment provisions supported changes, while AWS infrastructure outside the application model remains in the CDK workflow | **Encore.ts:** Encore Cloud environments hold deployment properties such as capacity, networking, and backups. Logical resource properties remain with the declaration. Sensitive application values use Encore [TypeScript secrets](https://encore.dev/docs/ts/primitives/secrets) or [Go secrets](https://encore.dev/docs/go/primitives/secrets). **Encore.go:** Encore Cloud environments hold deployment properties such as capacity, networking, and backups. Logical resource properties remain with the declaration. Sensitive application values use Encore [TypeScript secrets](https://encore.dev/docs/go/primitives/secrets) or [Go secrets](https://encore.dev/docs/go/primitives/secrets). ## Decide what remains in CDK CDK and Encore can be used together. CDK commonly continues to manage: - AWS account and organization configuration - VPCs, DNS, and shared networking - infrastructure without an Encore primitive - resources shared with non-Encore systems - custom resources and organization-specific constructs - infrastructure whose topology is generated during CDK synthesis The Encore platform can provision supported application resources while CDK retains those areas. The application connects to CDK-managed infrastructure using the relevant AWS SDK or client. Store any sensitive connection details in Encore secrets. For self-hosted deployments, CDK can provision all AWS infrastructure. An infra config file maps supported Encore declarations to those physical resources. See [Configure infrastructure](https://encore.dev/docs/self-host/configure-infra). ## CloudFormation ownership and existing resources CDK deploys through CloudFormation, so CloudFormation stack ownership remains relevant during migration. Encore does not read a CDK cloud assembly or adopt a CloudFormation stack as its application model. The Encore platform provides resource-specific workflows for connecting supported existing infrastructure: - [AWS RDS](https://encore.dev/docs/platform/infrastructure/aws/import-rds) - [Amazon S3](https://encore.dev/docs/platform/infrastructure/aws/import-s3-bucket) - [Amazon SNS](https://encore.dev/docs/platform/infrastructure/aws/import-sns-topic) This differs from `cdk import`. CDK import brings an existing AWS resource under CloudFormation management in a CDK stack. Encore's import workflow connects an Encore declaration to existing physical infrastructure. It does not convert the CDK construct tree into Encore declarations. Avoid managing the same physical resource through an active CloudFormation stack and Encore. Before moving ownership, set the CDK resource's removal policy to `RETAIN` and deploy that change. Then remove the construct and deploy again so CloudFormation stops managing the resource without deleting it, as described in the [CDK import guidance](https://docs.aws.amazon.com/cdk/v2/guide/cli.html#cli-import). Follow the relevant Encore import guide and verify the Encore environment before completing the handoff. If CDK retains ownership, leave the construct and CloudFormation resource in place. Treat the resource as an external dependency of the Encore application. ## Migrate incrementally 1. Choose one service and list the constructs and CloudFormation resources it depends on. 2. Map supported databases, Pub/Sub, object storage, caches, cron jobs, and secrets to [TypeScript primitives](https://encore.dev/docs/ts) or [Go primitives](https://encore.dev/docs/go). 3. Identify shared constructs, cross-stack references, custom resources, and unsupported AWS services that must remain in CDK. 4. Decide whether the target environment will use the Encore platform or be self-hosted. 5. Add the Encore declarations and test with `encore run` and `encore test`. 6. For the Encore platform, provision new resources or follow the supported import workflow. For self-hosting, retain CDK provisioning and map the resources through infra config. 7. Deploy and verify resource connectivity, data, IAM behavior, and rollback procedures. 8. Change CloudFormation ownership only after the Encore environment is verified. Keep CloudFormation outputs and cross-stack references stable until all consumers have moved. Repeat the process for the next service. ## Workflow differences Encore validates statically discoverable resource declarations during the build and configures their deployment per environment. It does not replace CDK synthesis, CloudFormation stacks, or `cdk diff`. Keep unsupported, shared, and dynamically generated infrastructure in CDK, and migrate one ownership boundary at a time. See [Development workflow](https://encore.dev/docs/platform/workflow) for local, preview, and production environments. For service-by-service application migration patterns, see [Migrating an existing system to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore). --- Source: https://encore.dev/docs/platform/migration/from-paas Section: Guides # Coming from a PaaS > Map application hosting and managed services to Encore A PaaS typically combines application hosting, managed infrastructure, and platform-specific services. Encore is a backend development platform: you define APIs and services with its TypeScript or Go SDK and declare the infrastructure they use in the same application code. Encore uses those declarations to build an [application model](https://encore.dev/docs/understanding-encore) of your services, APIs, resources, and their relationships. Encore applications can run in your own AWS or GCP account through the Encore platform, or on [Encore Cloud Hosting](https://encore.dev/docs/platform/infrastructure/infra#encore-cloud-hosting) for development and preview environments, or as [self-hosted Docker images](https://encore.dev/docs/self-host/build). An Encore service is an application-code boundary containing API endpoints and resource usage. Infrastructure primitives are resources, such as databases or Pub/Sub topics, declared through the SDK. The table below links to TypeScript examples; equivalent primitives are available in [Go](https://encore.dev/docs/go). ## How the concepts map | PaaS concept | Encore equivalent | |---|---| | Application or service | An Encore application containing one or more services | | Managed Postgres | A SQL database primitive | | Redis or managed cache | A cache for supported operations, or an external Redis service | | Object storage | An object storage bucket primitive | | Event-driven asynchronous work | Pub/Sub topics and subscriptions | | Scheduled jobs | Cron jobs | | Secret environment variables | Secrets with values set per environment | | Non-secret environment variables | No direct Encore primitive; use application constants for values that do not vary by environment or an external configuration service for runtime configuration | | Logs, traces, and service information | Encore platform [tracing](https://encore.dev/docs/tracing), [metrics](https://encore.dev/docs/platform/observability/metrics), structured logging, and the [service catalog](https://encore.dev/docs/service-catalog) | | Hosted authentication, generated APIs, realtime features, and add-ons | Application code, another service, or an external dependency | These primitives are declared in application code using [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go). Platform-specific features may need to be reimplemented, replaced, or kept as external services. ## Choose where the application runs Encore supports three deployment models: - **Encore Cloud Hosting:** Encore runs the application containers and supporting infrastructure on Encore-managed infrastructure. It does not require your own cloud account and is intended for development and preview environments, subject to [usage limits](https://encore.dev/docs/platform/management/usage). - **Your AWS or GCP account:** The Encore platform deploys the application and provisions supported resources in an account you own, using the environment's [infrastructure configuration](https://encore.dev/docs/platform/infrastructure/configuration). - **Self-hosted:** You deploy the image produced by `encore build docker`, provision the physical infrastructure, and connect the declarations to it using an [infra config file](https://encore.dev/docs/self-host/configure-infra). Cloud development and preview environments use the Encore platform Hosting by default. Hosting preview environments in your own cloud is available on the Enterprise tier. Self-hosted deployments configure their own metrics and logging destinations through the infra config and deployment environment. ## Migrate data and platform services ### Postgres databases Encore databases use Postgres and apply schema changes from migration files. For an existing database, choose between importing the current instance and transferring the data to a new instance. The Encore platform can connect directly to supported existing [AWS RDS](https://encore.dev/docs/platform/infrastructure/aws/import-rds), [Google Cloud SQL](https://encore.dev/docs/platform/infrastructure/gcp/import-cloud-sql), and [Neon](https://encore.dev/docs/platform/infrastructure/neon#import-an-existing-neon-branch) databases. When importing RDS or Cloud SQL with a schema not previously managed by Encore, follow the bootstrap migration procedure in the relevant import guide. Existing Neon databases may require role ownership changes before Encore can run migrations. Import a supported database when it should remain in place. Transfer the schema and data when the provider is unsupported or when the Encore platform should provision the target. For self-hosted deployments, the infra config can map the application declaration to an existing Postgres database. When transferring a database, plan version compatibility, extensions, roles, connection cutover, and any required downtime. ### Platform-specific services Hosted authentication, generated data APIs, realtime subscriptions, edge or serverless functions, persistent volumes, and marketplace add-ons do not always have direct Encore equivalents. For each feature, choose whether to: - implement it in an Encore service - replace it with an Encore primitive or another provider - continue using it as an external dependency Migrate these features independently; they do not all need to move at once. An external dependency remains outside Encore's provisioning lifecycle. Application code uses the provider's normal client or SDK, sensitive credentials are stored as Encore secrets, and the existing platform or infrastructure tooling continues to manage the service. ## Deployments and preview environments After connecting a GitHub repository, pushes to configured branches can trigger deployments. The Encore platform Pro can also create an ephemeral [preview environment](https://encore.dev/docs/platform/deploy/preview-environments) for each pull request. See [Deploying applications](https://encore.dev/docs/platform/deploy/deploying) for the available deployment methods. Infrastructure running in your AWS or GCP account is billed by the cloud provider. The Encore platform usage depends on the features and pricing of your [Encore plan](https://encore.dev/pricing). ## Plan the migration 1. Inventory application services, data stores, scheduled work, add-ons, secrets, domains, and network dependencies. 2. Map supported infrastructure to Encore primitives and identify platform services that need separate migration work. 3. Create an Encore application and move one service at a time, or wrap the existing HTTP router in a catch-all handler. See [Migrating an existing system to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore) for both approaches. 4. Validate the application locally and in a development or preview environment. 5. Import or migrate stateful resources and verify schema and data compatibility. 6. Switch traffic using the existing [custom domain](https://encore.dev/docs/platform/integrations/custom-domains) or gateway after the new environment is ready. 7. Keep the previous deployment available until the Encore environment has been verified. The cutover strategy and downtime requirements depend on the application and its data migration. ## Platform-specific guides - [Supabase](https://encore.dev/docs/platform/migration/from-supabase): migrate Postgres data and plan replacements for Auth, Storage, Realtime, and Edge Functions. - [Firebase](https://encore.dev/docs/platform/migration/from-firebase): migrate document data and plan replacements for Authentication, Security Rules, Storage, and Functions. - [Railway](https://encore.dev/docs/platform/migration/from-railway): map services, variables, databases, volumes, and cron jobs. - [Heroku](https://encore.dev/docs/platform/migration/from-heroku): map dynos, process types, config vars, add-ons, and Review Apps. - [Vercel](https://encore.dev/docs/platform/migration/from-vercel): move backend functions while keeping a Vercel-hosted frontend when appropriate. - [Fly.io](https://encore.dev/docs/platform/migration/from-flyio): map Machines, volumes, private networking, and managed services. ## Next steps - Follow the [Quickstart guide](https://encore.dev/docs/quick-start) to evaluate the local workflow. - Then choose between [the Encore platform deployment](https://encore.dev/docs/platform/deploy/deploying) and [self-hosting](https://encore.dev/docs/self-host/build). --- Source: https://encore.dev/docs/platform/migration/from-supabase Section: Guides # Coming from Supabase > Move a Supabase backend one part at a time [Supabase combines Postgres with generated data APIs, authentication, object storage, Realtime, and Edge Functions](https://supabase.com/docs/guides/getting-started/architecture). Moving to Encore means defining the backend operations your clients use with Encore's TypeScript or Go SDK, with databases, buckets, and other supported infrastructure declared alongside them. These declarations form the [application model](https://encore.dev/docs/understanding-encore) Encore uses for local development and deployment. You do not need to migrate every Supabase service at once. An Encore application can connect to a Supabase project through its normal APIs or Postgres connection string while you move individual workloads. ## How the concepts map | Supabase | Encore | |---|---| | Postgres database and SQL migrations | A SQL database and migration files | | REST Data API served by PostgREST | Explicit, type-safe API endpoints that query the database | | Row Level Security (RLS) for Data API access | Authorization in API handlers and database queries; Postgres RLS can still be used when required | | Supabase Auth | Keep Supabase Auth as an external service, or use another identity provider with an Encore authentication handler | | Storage Files buckets and Storage policies | Object storage buckets, with access controlled through Encore APIs | | Realtime database changes, Broadcast, and Presence | No single equivalent; use streaming APIs, Pub/Sub, or keep Supabase Realtime | | Edge Functions | Encore API endpoints, Pub/Sub subscribers, or cron jobs, depending on how the function is invoked | | Project secrets | Encore secrets, with values set per environment | | Supabase dashboard logs | Encore platform [traces](https://encore.dev/docs/tracing), [metrics](https://encore.dev/docs/platform/observability/metrics), and structured logs | These primitives are declared in application code using [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go). ## Choose what to move Start by recording which Supabase features the application uses directly from a browser or mobile client. Calls to the Data API, Auth, Storage, and Realtime often share Supabase access tokens and RLS or Storage policies. Moving one of those features may therefore require a new backend endpoint or a change to the client. For each feature, choose one of three approaches: - keep it in Supabase and call it from Encore as an external dependency - move it to an Encore primitive or application service - replace it with another provider and validate its credentials in an Encore authentication handler The [general PaaS migration guide](https://encore.dev/docs/platform/migration/from-paas) covers Encore's deployment models and the migration sequence shared by hosted platforms. ## Database and generated APIs A Supabase project uses Postgres, so its tables, indexes, constraints, functions, and data can be transferred with standard Postgres tools. Supabase documents which connection type to use for [`pg_dump` and other native commands](https://supabase.com/docs/guides/database/connecting-to-postgres). Encore does not directly import a Supabase-hosted database as a managed resource. To move the database: 1. Export the application schema and data. Exclude Supabase-managed schemas that belong to services you are not migrating. 2. Turn the application schema into an initial Encore migration and add the SQL database declaration. 3. Test the migrations against a new local database with `encore run`. 4. Create the target database from those migrations, then load the exported data without recreating the schema. 5. Check Postgres versions, extensions, roles, ownership, sequences, and functions before cutover. 6. Stop writes or replicate changes while performing the final data transfer, then switch the application to the new environment. Supabase-managed schemas such as `auth` and `storage` belong to their respective Supabase services. Do not treat them as application tables unless the replacement service explicitly supports that data and schema. **Encore.ts:** Encore does not generate a CRUD API from the database schema. Supabase no longer exposes one automatically either: since May 2026 new projects require tables to be granted to the API roles explicitly, and [that becomes the default for existing projects on 30 October 2026](https://supabase.com/changelog/45329-breaking-change-tables-not-exposed-to-data-and-graphql-api-automatically), so how much of the Data API there is to move depends on when the project was created. Define the operations your clients need as Encore endpoints and move queries behind those endpoints. [Generated TypeScript clients](https://encore.dev/docs/ts/cli/client-generation) can provide typed access from a frontend. **Encore.go:** Encore does not generate a CRUD API from the database schema. Supabase no longer exposes one automatically either: since May 2026 new projects require tables to be granted to the API roles explicitly, and [that becomes the default for existing projects on 30 October 2026](https://supabase.com/changelog/45329-breaking-change-tables-not-exposed-to-data-and-graphql-api-automatically), so how much of the Data API there is to move depends on when the project was created. Define the operations your clients need as Encore endpoints and move queries behind those endpoints. [Generated TypeScript clients](https://encore.dev/docs/go/cli/client-generation) can provide typed access from a frontend. If clients currently access Supabase directly, run the new API alongside the Data API and migrate one operation at a time. Keep RLS policies in place for traffic that still reaches PostgREST. For the new Encore endpoints, reproduce the required authorization checks in application code or SQL; do not assume existing RLS policies and backend credentials enforce the same access path. ## Authentication **Encore.ts:** Encore provides the authentication hook used by its API gateway, but it does not provide a hosted user directory or sign-in UI. You can keep Supabase Auth during and after the migration by validating access tokens according to [Supabase's JWT guidance](https://supabase.com/docs/guides/auth/jwts) in an Encore [authentication handler](https://encore.dev/docs/ts/develop/auth). The existing client sign-in and session-refresh flow can then remain in place while Encore protects its own endpoints. **Encore.go:** Encore provides the authentication hook used by its API gateway, but it does not provide a hosted user directory or sign-in UI. You can keep Supabase Auth during and after the migration by validating access tokens according to [Supabase's JWT guidance](https://supabase.com/docs/guides/auth/jwts) in an Encore [authentication handler](https://encore.dev/docs/go/develop/auth). The existing client sign-in and session-refresh flow can then remain in place while Encore protects its own endpoints. If you replace Supabase Auth, plan the migration with the destination identity provider. Supabase stores user records in the `auth` schema, but copying those tables is not a supported substitute for importing users through the destination provider. Check how password hashes, social identities, MFA factors, email verification, custom claims, and active sessions are handled. A staged migration may require users to sign in again or reset their passwords. Keep user identifiers stable where possible. If identifiers change, migrate foreign keys and application records that refer to `auth.users.id` before switching authentication providers. ## Storage Encore object storage provides bucket operations and signed URLs, but it does not reproduce Supabase Storage policies or its database metadata. To move a bucket: **Encore.ts:** 1. Declare the target [object storage bucket](https://encore.dev/docs/ts/primitives/object-storage). 2. Copy the objects with an object-storage transfer tool, preserving object paths and required metadata. This applies to Files buckets. Supabase Analytics buckets (Apache Iceberg) and Vector buckets have no Encore equivalent and need a separate destination. 3. Replace Supabase Storage client calls and URLs with Encore API calls or URLs generated for the new bucket. 4. Implement access checks in the Encore endpoints that issue signed URLs or read and write objects. 5. Update stored public URLs, cache rules, CORS settings, and upload limits where applicable. **Encore.go:** 1. Declare the target [object storage bucket](https://encore.dev/docs/go/primitives/object-storage). 2. Copy the objects with an object-storage transfer tool, preserving object paths and required metadata. This applies to Files buckets. Supabase Analytics buckets (Apache Iceberg) and Vector buckets have no Encore equivalent and need a separate destination. 3. Replace Supabase Storage client calls and URLs with Encore API calls or URLs generated for the new bucket. 4. Implement access checks in the Encore endpoints that issue signed URLs or read and write objects. 5. Update stored public URLs, cache rules, CORS settings, and upload limits where applicable. Existing signed URLs are tied to Supabase and should not be expected to work after the objects move. Keep the old bucket available until clients and persisted URLs have been updated. ## Edge Functions and background work **Encore.ts:** Move HTTP Edge Functions to Encore API endpoints. A function invoked by a webhook can usually become a public [raw endpoint](https://encore.dev/docs/ts/primitives/raw-endpoints), while authenticated application operations can use typed endpoints. **Encore.go:** Move HTTP Edge Functions to Encore API endpoints. A function invoked by a webhook can usually become a public [raw endpoint](https://encore.dev/docs/go/primitives/raw-endpoints), while authenticated application operations can use typed endpoints. Use Pub/Sub subscribers for asynchronous work and cron jobs for schedules. Database webhooks and triggers need separate treatment: publish an event from the application when Encore owns every write path, or retain a database-level integration when other systems can still modify the table. Confirm retry, ordering, timeout, and idempotency behavior rather than assuming it matches the Edge Function deployment. Supabase Edge Functions run in a Deno-compatible runtime. Encore TypeScript services run on Node.js, so review Deno-specific APIs, imports, permissions, and runtime assumptions during the rewrite. ## Realtime [Supabase Realtime](https://supabase.com/docs/guides/realtime) covers three different capabilities: Postgres change subscriptions, Broadcast channels, and Presence. Encore does not provide a drop-in replacement for that combined service. - Use [streaming APIs](https://encore.dev/docs/ts/primitives/streaming-apis) (TypeScript) for application-managed WebSocket connections. - Use [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub) for events exchanged between backend services. Pub/Sub is not a browser subscription API by itself. - Keep Supabase Realtime when clients still need Postgres change feeds, Broadcast, or Presence, and connect it to the remaining Supabase database. When replacing database change subscriptions, define the events the client actually needs instead of exposing every row change. Account for reconnects, missed messages, authorization, and the initial data load as part of the client protocol. ## Run Supabase and Encore together **Encore.ts:** Store any [secret API key](https://supabase.com/docs/guides/getting-started/api-keys), legacy `service_role` key, or database connection string as an Encore [secret](https://encore.dev/docs/ts/primitives/secrets). Use elevated keys only in backend code and only for operations that need to bypass RLS. For user-scoped Supabase requests, pass the user's access token through a client initialized with a publishable key instead. **Encore.go:** Store any [secret API key](https://supabase.com/docs/guides/getting-started/api-keys), legacy `service_role` key, or database connection string as an Encore [secret](https://encore.dev/docs/go/primitives/secrets). Use elevated keys only in backend code and only for operations that need to bypass RLS. For user-scoped Supabase requests, pass the user's access token through a client initialized with a publishable key instead. An incremental migration can follow this order: 1. Deploy an Encore service without changing the existing Supabase clients. 2. Validate Supabase tokens in Encore and move selected business operations behind Encore APIs. 3. Move Edge Functions and background work whose data dependencies are understood. 4. Decide whether to retain or replace Auth, Storage, and Realtime independently. 5. Transfer the Postgres database after all remaining direct client access and replication requirements are accounted for. 6. Switch traffic, monitor both systems, and remove Supabase resources only after data and client behavior have been verified. Avoid concurrent writes to old and new databases unless you have designed conflict handling and reconciliation. The required downtime depends on the transfer method and the application's write volume. Supabase services kept during the migration remain in the Supabase project's provisioning and billing lifecycle. Encore secrets and external-dependency code provide the connection; they do not transfer ownership of those services to Encore. ## Before cutover - Compare row counts and application-level invariants after the final database copy. - Exercise sign-in, token refresh, authorization, and account-recovery flows. - Verify object counts, metadata, access checks, and persisted URLs. - Test Realtime reconnect behavior and delivery after a deployment or network interruption. - Check webhook retries, scheduled jobs, and idempotency. - Update client configuration, CORS, custom domains, and callback URLs. - Keep a rollback path until the new deployment has handled production traffic successfully. ## Next steps - Follow the [Quickstart guide](https://encore.dev/docs/quick-start). - Read [Migrating an existing system to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore) for service-by-service adoption patterns. - Review [Deploying applications](https://encore.dev/docs/platform/deploy/deploying) before choosing a production environment. --- Source: https://encore.dev/docs/platform/migration/from-firebase Section: Guides # Coming from Firebase > Move a Firebase backend one part at a time Firebase provides several independent backend products, including Cloud Firestore, Realtime Database, SQL Connect, Authentication, Cloud Storage, and Cloud Functions. Moving to Encore usually replaces direct client access with APIs defined using Encore's TypeScript or Go SDK, with authorization and data access handled in the backend. Those APIs and their infrastructure declarations form the [application model](https://encore.dev/docs/understanding-encore) Encore uses for local development and deployment. There is no single Firebase project import. Firestore and Realtime Database use NoSQL data models, SQL Connect combines Cloud SQL for PostgreSQL with a managed GraphQL API, native SQL operations, and realtime query subscriptions, Firebase Authentication is an identity service, and Cloud Storage stores objects in Google Cloud Storage. Plan and validate each migration separately. ## How the concepts map | Firebase | Encore | |---|---| | Cloud Firestore collections and documents | Usually a relational schema in an Encore SQL database; there is no direct document-database primitive | | Realtime Database JSON tree | An application-specific SQL schema or an external database | | Firebase SQL Connect schema, operations, and generated SDKs | An Encore SQL database and explicit API endpoints; the existing Cloud SQL instance can also remain external | | Client SDK queries and Security Rules | Explicit API endpoints, with authorization and validation in backend code | | Firebase Authentication | Keep Firebase Authentication as an external service, or use another identity provider with an Encore authentication handler | | Cloud Storage for Firebase | An Encore object storage bucket | | Callable and HTTP Cloud Functions | Encore API endpoints | | Firestore, Auth, and Storage triggers | Pub/Sub subscribers or provider-specific integrations | | Scheduled functions | Cron jobs | | Firestore and Realtime Database listeners | No direct equivalent; use streaming APIs and application-defined events, or retain Firebase | | Cloud Functions secrets and other sensitive configuration | Encore secrets, with values set per environment | These primitives are declared in application code using [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go). Firebase Hosting, App Hosting, Remote Config, Cloud Messaging, Analytics, Crashlytics, and App Check are separate products. They can remain in use where appropriate; [Firebase Extensions](https://firebase.google.com/docs/extensions) is the exception, having been deprecated with a shutdown date of 31 March 2027, so plan its replacement alongside this migration. Declaring an Encore service does not migrate or replace any of them. If App Check protects a client operation you move, the client can [send its App Check token to the new backend](https://firebase.google.com/docs/app-check/web/custom-resource), where the endpoint verifies it with the Firebase Admin SDK. ## Choose what to move Inventory the Firebase SDK calls made by browser and mobile clients. Direct Firestore and Realtime Database access depends on Security Rules, local caching, and realtime listeners. Replacing those calls with Encore APIs changes both the trust boundary and, potentially, offline behavior. For each Firebase product, choose whether to: - keep it and call it from Encore through the Firebase Admin SDK or its Google Cloud API - replace it with an Encore primitive or application service - move it to another provider and integrate that provider with Encore The [general PaaS migration guide](https://encore.dev/docs/platform/migration/from-paas) covers Encore's deployment models and the migration sequence shared by hosted platforms. ## Firestore Firestore exports contain documents and collection structure; they are not Postgres dumps. Moving the data to an Encore SQL database requires a relational schema and a transformation step. Plan the migration as follows: **Encore.ts:** 1. List the collections, subcollections, document references, indexes, and queries used by the application. 2. Design SQL tables, keys, relationships, and indexes for those access patterns. 3. Add [database migrations](https://encore.dev/docs/ts/primitives/databases#database-migrations) and explicit Encore endpoints for the required operations. 4. [Export Firestore data to Cloud Storage](https://firebase.google.com/docs/firestore/manage-data/export-import) using its managed export service, then write a repeatable transformation and load process for the new schema. 5. Validate document counts as well as application-level relationships and aggregates. 6. Account for writes that occur after the export before switching traffic. **Encore.go:** 1. List the collections, subcollections, document references, indexes, and queries used by the application. 2. Design SQL tables, keys, relationships, and indexes for those access patterns. 3. Add [database migrations](https://encore.dev/docs/go/primitives/databases#database-migrations) and explicit Encore endpoints for the required operations. 4. [Export Firestore data to Cloud Storage](https://firebase.google.com/docs/firestore/manage-data/export-import) using its managed export service, then write a repeatable transformation and load process for the new schema. 5. Validate document counts as well as application-level relationships and aggregates. 6. Account for writes that occur after the export before switching traffic. A managed Firestore export is not an exact snapshot taken at the start of the operation and may include changes made while it runs. Use a maintenance window, a change-capture process, or another reconciliation strategy if the application must remain writable. Firestore transactions, atomic field operations, server timestamps, geospatial conventions, and denormalized documents need explicit SQL equivalents. Test query semantics and contention behavior rather than translating documents mechanically. ## SQL Connect [Firebase SQL Connect](https://firebase.google.com/docs/sql-connect) uses Cloud SQL for PostgreSQL together with a managed GraphQL API, native SQL operations, and generated client SDKs. It also offers [realtime query subscriptions](https://firebase.google.com/docs/sql-connect/realtime) and client-side caching, which have no Encore equivalent and need replacing rather than porting. You can keep SQL Connect as an external dependency, or move its operations to Encore endpoints and manage the PostgreSQL schema through Encore migrations. When deploying with the Encore platform on GCP, a supported existing Cloud SQL instance can be connected through the [Cloud SQL import workflow](https://encore.dev/docs/platform/infrastructure/gcp/import-cloud-sql). Follow that guide's bootstrap-migration procedure for an existing schema, and choose one migration system to own future schema changes. If SQL Connect must continue managing the schema, leave the database external instead of applying Encore migrations to the same tables. ## Realtime Database Realtime Database stores data as a JSON tree. You can [export a node and its children as JSON](https://firebase.google.com/docs/database/manage-data-console), then transform that JSON into the new database schema. Review multi-location updates, generated keys, transactions, priorities, and denormalized paths before importing the data. A successful JSON load does not by itself preserve listener behavior or Security Rules. If the application relies on offline persistence and automatic client synchronization from Firestore or Realtime Database, plan that feature separately. Encore endpoints and streaming APIs do not provide a drop-in offline database client. ## Security Rules and API access Firebase web and mobile SDKs can access Firestore, Realtime Database, and Cloud Storage directly, with [Firebase Security Rules](https://firebase.google.com/docs/rules) enforcing access. Server Firebase libraries instead use Google Cloud IAM or administrative credentials and can bypass those client rules. With Encore, clients call API endpoints. The endpoint authenticates the caller, validates the request, performs authorization, and accesses the resource from backend code. Translate each relevant rule into tests and application checks before moving a client operation. Pay particular attention to ownership conditions, custom claims, field-level validation, query constraints, and rules that read other documents. During coexistence, keep Security Rules for direct Firebase client traffic. Encore code using administrative credentials must enforce its own authorization; do not rely on the client rules to protect that path. ## Authentication **Encore.ts:** Encore provides the authentication hook used by its API gateway, but it does not provide a hosted user directory or sign-in UI. You can keep Firebase Authentication by verifying Firebase ID tokens in an Encore [authentication handler](https://encore.dev/docs/ts/develop/auth). Existing client sign-in flows can then continue while Encore protects its endpoints. **Encore.go:** Encore provides the authentication hook used by its API gateway, but it does not provide a hosted user directory or sign-in UI. You can keep Firebase Authentication by verifying Firebase ID tokens in an Encore [authentication handler](https://encore.dev/docs/go/develop/auth). Existing client sign-in flows can then continue while Encore protects its endpoints. If you move to another identity provider, [Firebase CLI can export accounts as JSON or CSV](https://firebase.google.com/docs/cli/auth). Password migration depends on whether the destination supports Firebase's password hash and parameters. Firebase exports only SCRYPT-backed password hashes; records originally imported with another algorithm can have empty hash fields until the user signs in and Firebase rehashes the password. Social-provider links, phone users, MFA enrollment, custom claims, email verification, and active sessions also require a provider-specific migration plan. Keep Firebase UIDs as the destination subject identifier where possible. If identifiers change, update every Firestore document, Realtime Database path, SQL row, Storage rule, and application record that refers to the UID. Some migrations require users to sign in again or reset their passwords. ## Cloud Storage Cloud Storage for Firebase stores objects in Google Cloud Storage. You can keep the bucket as an external dependency, copy its objects to a new bucket declared with Encore, or connect a declared bucket to the existing GCS bucket through the Encore platform's [GCS import workflow](https://encore.dev/docs/platform/infrastructure/gcp/import-gcs-bucket). Access through Google Cloud APIs or Encore uses Google Cloud IAM rather than Firebase Authentication and Storage Rules. Recreate the required authorization in your Encore APIs even when the objects remain in the same bucket. When moving objects: **Encore.ts:** 1. Declare the target [object storage bucket](https://encore.dev/docs/ts/primitives/object-storage). 2. Copy objects with [Google Cloud Storage tools](https://firebase.google.com/docs/storage/gcp-integration) or a managed transfer service, preserving paths and required metadata. 3. Replace Firebase Storage SDK calls and download URLs in clients. 4. Implement the former Storage Rules as authorization checks in the Encore APIs that read, write, or issue signed URLs. 5. Verify CORS, cache metadata, content types, object generations, and upload limits. **Encore.go:** 1. Declare the target [object storage bucket](https://encore.dev/docs/go/primitives/object-storage). 2. Copy objects with [Google Cloud Storage tools](https://firebase.google.com/docs/storage/gcp-integration) or a managed transfer service, preserving paths and required metadata. 3. Replace Firebase Storage SDK calls and download URLs in clients. 4. Implement the former Storage Rules as authorization checks in the Encore APIs that read, write, or issue signed URLs. 5. Verify CORS, cache metadata, content types, object generations, and upload limits. Firebase download tokens and URLs should not be expected to remain valid for copied objects. Keep the original bucket available until clients and stored URLs have been updated. ## Cloud Functions Map each function according to its trigger: - Move HTTP and callable functions to Encore API endpoints. Callable functions use a Firebase-specific request and response protocol, so update clients rather than exposing an Encore endpoint at the old URL without a compatibility layer. - Move scheduled functions to Encore cron jobs. - Move asynchronous application events to Pub/Sub topics and subscriptions. - Keep Firebase or Google Cloud event integrations for Firestore, Realtime Database, Authentication, and Storage events that remain outside Encore. If Encore becomes the only writer for a workflow, it can publish an application event after a successful operation. If Firebase clients or other systems still write directly, publishing only from Encore would miss those changes. Cloud Functions run in a Google-managed functions environment. Review runtime versions, request formats, environment configuration, concurrency, timeouts, retries, IAM, regional placement, and temporary filesystem assumptions when moving the code. Make event handlers idempotent before enabling retries in the replacement workflow. ## Realtime clients Encore [streaming APIs](https://encore.dev/docs/ts/primitives/streaming-apis) (TypeScript) provide WebSocket streams between a client and an API handler. [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub) distributes events between backend services. Together they can support application-defined updates, but they do not reproduce Firestore queries, local caches, Security Rules, or Realtime Database synchronization. Define the client protocol explicitly: the initial data load, which changes produce events, authorization per connection, ordering, reconnect behavior, and recovery from missed messages. Retaining Firebase listeners while other parts move to Encore is often the smaller first step. ## Run Firebase and Encore together **Encore.ts:** Store Firebase service-account credentials or other sensitive configuration in Encore [secrets](https://encore.dev/docs/ts/primitives/secrets), and use the Firebase Admin SDK only from backend code. Grant the service account only the permissions the service requires. **Encore.go:** Store Firebase service-account credentials or other sensitive configuration in Encore [secrets](https://encore.dev/docs/go/primitives/secrets), and use the Firebase Admin SDK only from backend code. Grant the service account only the permissions the service requires. An incremental migration can follow this order: 1. Deploy an Encore service and keep existing Firebase clients unchanged. 2. Verify Firebase ID tokens in Encore and move selected operations behind Encore APIs. 3. Move HTTP functions and backend-only workflows. 4. Move client data access one query or workflow at a time, translating its Security Rules into backend authorization. 5. Migrate Storage, Authentication, and realtime behavior independently if they need to move. 6. Perform the final data transfer or reconciliation, switch traffic, and monitor both systems. Avoid writing independently to Firebase and the replacement database unless you have designed conflict handling and reconciliation. Keep the old system available until the new data, authorization, and client behavior have been verified. Firebase products retained during the migration remain in the Firebase and Google Cloud provisioning and billing lifecycle. Encore secrets and application code provide the integration; they do not transfer ownership of those resources to Encore. Firebase Hosting can continue serving a frontend that calls an Encore API. Update its runtime configuration, rewrites, CORS policy, and custom-domain routing as required by the new API origin. ## Before cutover - Compare document or node counts with the loaded rows and check application-level invariants. - Test sign-in, token refresh, authorization, account recovery, and any custom claims. - Verify Storage object counts, metadata, access checks, and persisted URLs. - Exercise listener reconnects, missed-event recovery, and offline behavior. - Check function retries, schedules, webhooks, and idempotency. - Update client configuration, CORS, callback URLs, Security Rules, and service-account permissions. - Keep a rollback path until the new deployment has handled production traffic successfully. ## Next steps - Follow the [Quickstart guide](https://encore.dev/docs/quick-start). - Read [Migrating an existing system to Encore](https://encore.dev/docs/platform/migration/migrate-to-encore) for service-by-service adoption patterns. - Review [Deploying applications](https://encore.dev/docs/platform/deploy/deploying) before choosing a production environment. --- Source: https://encore.dev/docs/platform/migration/from-railway Section: Guides # Coming from Railway > Move services and attached resources incrementally Railway treats each deployed service and attached resource as a platform unit. In Encore, the APIs, services, and infrastructure declared with the TypeScript or Go SDK form an [application model](https://encore.dev/docs/understanding-encore). Encore uses that model to run the application locally and determine what each environment needs. You can migrate one service at a time. Railway and Encore may remain behind the same frontend or gateway while traffic is moved. ## How the concepts map | Railway | Encore | |---|---| | Project and environment | Encore application and [environment](https://encore.dev/docs/platform/deploy/environments) | | Service | One or more Encore services; the boundary is defined in code | | Deployment | A rollout of the Encore application | | PostgreSQL template | SQL database | | Redis template | Cache, when its API fits the workload, or an external Redis service | | Variables and references | Secrets, code constants, or an external configuration service | | Cron service | Cron job calling an Encore API endpoint | | Volume | No direct primitive; use object storage, a database, or externally managed persistent storage | | Public networking (Railway-provided domain) | Environment API URL or [custom domain](https://encore.dev/docs/platform/integrations/custom-domains) | These primitives are declared in application code using [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go). An Encore service is not a Railway deployment unit. Encore builds the service graph as one application and chooses the deployment layout from the environment configuration. ## Migrate services Move an HTTP service by defining its routes as Encore API endpoints. Existing TypeScript or Go business logic can remain in place. For a larger server, first expose the existing router through a catch-all endpoint, then convert routes incrementally; see [Migrating an existing system](https://encore.dev/docs/platform/migration/migrate-to-encore). **Encore.ts:** Railway worker services do not have a single equivalent. Model event-driven work with [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub), scheduled work with cron jobs, or keep a continuously running worker outside Encore if its lifecycle requires one. **Encore.go:** Railway worker services do not have a single equivalent. Model event-driven work with [Pub/Sub](https://encore.dev/docs/go/primitives/pubsub), scheduled work with cron jobs, or keep a continuously running worker outside Encore if its lifecycle requires one. Railway build and start commands, health-check paths, replicas, and restart policies do not carry over directly. Encore builds from the application model, and deployment-specific compute settings belong in [infrastructure configuration](https://encore.dev/docs/platform/infrastructure/configuration). ## Migrate data and configuration Encore SQL databases use Postgres migrations stored with the application. A Railway PostgreSQL database cannot be imported directly into the Encore platform. Create a target database and transfer its schema and data, accounting for extensions, roles, connection cutover, and downtime. A self-hosted deployment can map an Encore database declaration to an existing compatible Postgres server through its infra config. For Redis, check which commands and data structures the application uses before choosing Encore caching. The cache primitive is not a general-purpose Redis connection. Unsupported uses can remain on Railway temporarily and be accessed with the normal client library. Move sensitive Railway variables to Encore secrets and set a value for each environment. There is no general primitive for non-secret environment variables. Keep invariant values in code; use an external configuration service when values must change independently of a deployment. **Encore.ts:** Railway volumes are attached to a service and are not an Encore resource. Move shared files to [object storage](https://encore.dev/docs/ts/primitives/object-storage), structured state to a database, or retain external storage. Do not copy a live volume without an application-specific consistency plan. **Encore.go:** Railway volumes are attached to a service and are not an Encore resource. Move shared files to [object storage](https://encore.dev/docs/go/primitives/object-storage), structured state to a database, or retain external storage. Do not copy a live volume without an application-specific consistency plan. ## Deployments, domains, and cutover Connecting GitHub lets branch pushes trigger Encore deployments; Pro plans can create [preview environments](https://encore.dev/docs/platform/deploy/preview-environments) for pull requests. Railway configuration such as watch paths and per-service deployment triggers must be recreated using the Encore deployment workflow or your CI system. Create and verify the Encore environment before changing DNS. Add the domain in Encore, update DNS as described in [Custom Domains](https://encore.dev/docs/platform/integrations/custom-domains), and keep the Railway deployment available during verification. For a staged migration, route only selected paths or callers to Encore. ## Migration plan 1. Inventory Railway services, databases, variables, volumes, domains, and private-network connections. 2. Select one stateless HTTP service and move it to Encore without changing its external contract. 3. Replace Railway variable references with secrets or explicit external configuration. 4. Migrate stateful resources separately and test data compatibility. 5. Validate the service locally and in an Encore development or preview environment. 6. Shift traffic, monitor errors and traces, then remove the corresponding Railway service. Railway-specific features without an Encore equivalent can remain external. Store their credentials as secrets and manage them through Railway until they are replaced. For the concepts common to all hosted platforms, see [Coming from a PaaS](https://encore.dev/docs/platform/migration/from-paas). Railway behavior referenced here is documented in its guides for [services](https://docs.railway.com/services), [variables](https://docs.railway.com/variables), [volumes](https://docs.railway.com/volumes), [PostgreSQL](https://docs.railway.com/databases/postgresql), and [cron jobs](https://docs.railway.com/cron-jobs). --- Source: https://encore.dev/docs/platform/migration/from-heroku Section: Guides # Coming from Heroku > Move dynos and add-ons incrementally Heroku organizes an application around process types, dynos, config vars, and add-ons. Encore instead derives an [application model](https://encore.dev/docs/understanding-encore) from the APIs, services, and infrastructure declared with its TypeScript or Go SDK. The model describes the backend's structure and resource relationships rather than commands for runtime process types. ## How the concepts map | Heroku | Encore | |---|---| | App and pipeline stage | Encore application and [environment](https://encore.dev/docs/platform/deploy/environments) | | `web` process type | Encore API endpoints, grouped into services | | Worker dyno | Pub/Sub subscription, cron endpoint, or an externally operated worker | | Dyno formation and size | Environment [compute configuration](https://encore.dev/docs/platform/infrastructure/configuration) | | Heroku Postgres | SQL database | | Heroku Key-Value Store | Cache for supported cache use cases, or an external Redis- or Valkey-compatible store | | Config vars | Secrets, constants, or external configuration | | Scheduler | Cron jobs | | Add-on | Encore primitive, application code, or external dependency | | Review app | [Preview environment](https://encore.dev/docs/platform/deploy/preview-environments) | These primitives are declared in application code using [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go). An Encore service is a code and API boundary, not a dyno process type. Encore deploys the application from its service and resource graph; it does not read a `Procfile` or preserve a dyno formation. ## Migrate web and worker processes Define routes from a Heroku web process as Encore API endpoints. Existing TypeScript or Go application logic can remain behind those endpoints. A catch-all endpoint can wrap an existing router during an incremental migration; see [Migrating an existing system](https://encore.dev/docs/platform/migration/migrate-to-encore). Classify each non-web process by what triggers it: **Encore.ts:** - Use [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub) for event-driven background work. - Use [cron jobs](https://encore.dev/docs/ts/primitives/cron-jobs) for scheduled calls to API endpoints. - Keep an external worker when it requires an indefinite process, a protocol, or runtime behavior not represented by Encore. **Encore.go:** - Use [Pub/Sub](https://encore.dev/docs/go/primitives/pubsub) for event-driven background work. - Use [cron jobs](https://encore.dev/docs/go/primitives/cron-jobs) for scheduled calls to API endpoints. - Keep an external worker when it requires an indefinite process, a protocol, or runtime behavior not represented by Encore. Release-phase commands and one-off dynos have no general one-to-one mapping. Database migrations declared for an Encore SQL database run as part of deployment. Handle other administrative jobs through CI, a protected API endpoint, or an external operational process. ## Migrate databases, Key-Value Store, and add-ons Heroku Postgres cannot be imported directly into the Encore platform. Provision a target Postgres database and transfer the schema and data. Review extensions, ownership, roles, version compatibility, and the write cutover. For self-hosting, an infra config can connect an Encore database declaration to a compatible existing server. Encore's cache primitive does not expose the full Redis or Valkey API. Check the application's commands, persistence assumptions, and eviction behavior before replacing Heroku Key-Value Store. Applications that require the general-purpose API can continue using it as an external dependency. Treat each add-on as a separate migration. Replace it with an Encore primitive where there is a functional match, implement the capability in application code, move to another provider, or retain the add-on while Heroku permits access from the new environment. Check network allowlists and add-on attachment or billing rules before assuming it can be used outside the Heroku app. Move credentials from config vars to Encore secrets. Non-secret config vars do not have a direct equivalent: use constants for invariant values or an external configuration service for runtime configuration. ## Deployments, domains, and cutover Encore can deploy on pushes from a connected GitHub repository and create per-PR preview environments on supported plans. Heroku pipeline promotion, build configuration (classic buildpacks and slugs on Cedar, Cloud Native Buildpacks and OCI images on Fir), and `app.json` setup are not transferred; reproduce required build or release work in Encore or CI. Set up the Encore environment and verify its generated API URL first. Then add the existing hostname as an Encore [custom domain](https://encore.dev/docs/platform/integrations/custom-domains) and update DNS. Keep the Heroku app available until traffic and background work have been verified. If the application shares a database during the transition, both versions must be compatible with the active schema. ## Migration plan 1. Inventory process types, dyno formation, config vars, add-ons, domains, and scheduled jobs. 2. Move one stateless web process or bounded route group into Encore. 3. Convert workers according to their trigger and delivery requirements. 4. Migrate add-ons independently; do not treat the marketplace as one unit. 5. Transfer database data and rehearse the final write cutover. 6. Shift traffic, monitor the Encore deployment, and then scale down the replaced dynos. For shared PaaS migration guidance, see [Coming from a PaaS](https://encore.dev/docs/platform/migration/from-paas). Heroku behavior referenced here is documented in [The Procfile](https://devcenter.heroku.com/articles/procfile), [Config Vars](https://devcenter.heroku.com/articles/config-vars), [Heroku Postgres](https://devcenter.heroku.com/articles/heroku-postgresql), [Heroku Key-Value Store](https://devcenter.heroku.com/articles/heroku-redis), and [Review Apps](https://devcenter.heroku.com/articles/github-integration-review-apps). --- Source: https://encore.dev/docs/platform/migration/from-vercel Section: Guides # Coming from Vercel > Move backend workloads while keeping the frontend on Vercel Vercel can build a frontend and deploy backend code as Functions or as part of a multi-framework Vercel Services project. A common migration keeps the frontend on Vercel and moves its APIs and background workloads to Encore. In Encore, those workloads and their infrastructure are declared with the TypeScript or Go SDK and form an [application model](https://encore.dev/docs/understanding-encore) used for local development and deployment. ## How the concepts map | Vercel | Encore | |---|---| | Project environment | Encore [environment](https://encore.dev/docs/platform/deploy/environments) | | Vercel Service | One or more Encore services; the boundary is defined in application code | | Function or route handler | Encore API endpoint within a service | | Fluid compute configuration | Environment compute configuration; not a per-function declaration | | Cron Job | Cron job calling an API endpoint | | Marketplace Postgres integration | SQL database or an external database | | Marketplace KV or Redis integration | Cache for supported operations, or external Redis | | Vercel Queues (Beta) topic and consumer group | Pub/Sub topic and subscription, after reviewing delivery semantics | | Vercel Blob | Object storage or continued use as an external service | | Environment variable | Secret, constant, or external configuration | | Preview deployment | [Preview environment](https://encore.dev/docs/platform/deploy/preview-environments) | These primitives are declared in application code using [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go). An Encore endpoint is not deployed as an independently configured Vercel Function. Encore analyzes calls and resource usage across the application and deploys services according to the target environment. ## Move backend routes Define Vercel API routes, route handlers, or server actions that form a public backend contract as Encore API endpoints. For a Vercel Services project, map routes and service bindings before deciding which boundaries to preserve. Keep rendering and frontend-only server logic on Vercel when it benefits from the framework's request lifecycle. Update the frontend to call the Encore environment URL. For browser requests, configure the required CORS behavior and authentication explicitly. Do not move server actions mechanically: code coupled to framework rendering, cookies, or cache invalidation may be better left on Vercel, calling Encore as its backend. **Encore.ts:** Long-running or asynchronous work should not be mapped solely by function duration. Use [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub) when work is driven by an event and cron jobs when it is scheduled. When replacing Vercel Queues, review retention, retries, ordering, idempotency, and deployment isolation rather than treating its topics and consumer groups as exact equivalents. **Encore.go:** Long-running or asynchronous work should not be mapped solely by function duration. Use [Pub/Sub](https://encore.dev/docs/go/primitives/pubsub) when work is driven by an event and cron jobs when it is scheduled. When replacing Vercel Queues, review retention, retries, ordering, idempotency, and deployment isolation rather than treating its topics and consumer groups as exact equivalents. ## Move data and scheduled work Vercel storage products and marketplace integrations are separate services, so migration depends on the provider behind each integration. - For Postgres, either keep the provider as an external dependency or transfer data to an Encore database. The Encore platform can directly connect supported existing RDS, Cloud SQL, and [Neon](https://encore.dev/docs/platform/infrastructure/neon#import-an-existing-neon-branch) databases; other providers require a data transfer or self-hosted mapping. - Encore caching is not a full Redis interface. Confirm required commands and persistence behavior before replacing a KV or Redis integration. - Move Blob objects to an Encore bucket only after accounting for object keys, metadata, access URLs, upload flows, and consistency during cutover. Blob can remain external in the interim. - Recreate Vercel Cron Jobs as Encore cron declarations. Encore cron invokes an API endpoint; confirm the schedule, authentication assumptions, time zone, retries, and idempotency. Move sensitive environment variables to Encore secrets. Variables still needed by the frontend or its Vercel build remain in Vercel. Values exposed to browser bundles are not secrets and should not be copied into Encore merely to hide them. ## Deployments and domains The frontend and backend can use separate deployment workflows. Vercel can continue producing frontend preview deployments while Encore creates backend preview environments. If previews must be paired, pass the relevant Encore preview URL to the frontend build or use a stable development environment; this association is not automatic. Use an Encore [custom domain](https://encore.dev/docs/platform/integrations/custom-domains) for the backend, or proxy an API path from the frontend domain. Verify cookies, CORS, OAuth callback URLs, CSP rules, and any firewall allowlists before switching traffic. ## Migration plan 1. Separate frontend rendering concerns from backend APIs and background work. 2. Move one API route group to an Encore service and call it from the Vercel deployment. 3. Move only the environment variables required by that backend code. 4. Decide separately whether each database and storage integration stays external or migrates. 5. Recreate cron and asynchronous work, then test retries and duplicate delivery. 6. Pair preview deployments if required, verify the production domain, and shift API traffic. Vercel-specific features such as ISR, Routing Middleware, image optimization, and framework cache controls are frontend or platform concerns rather than Encore equivalents. They can remain on Vercel. For shared guidance, see [Coming from a PaaS](https://encore.dev/docs/platform/migration/from-paas). Vercel behavior referenced here is documented in [Functions](https://vercel.com/docs/functions), [Services](https://vercel.com/docs/services), [Queues](https://vercel.com/docs/queues), [Cron Jobs](https://vercel.com/docs/cron-jobs), [Environment Variables](https://vercel.com/docs/environment-variables), and [Storage](https://vercel.com/docs/storage). --- Source: https://encore.dev/docs/platform/migration/from-flyio Section: Guides # Coming from Fly.io > Move machine-based workloads and state incrementally Fly.io runs applications as individually configurable Machines. Encore starts from the APIs, services, and infrastructure declared with its TypeScript or Go SDK, using them to build an [application model](https://encore.dev/docs/understanding-encore). Machine-level settings in `fly.toml` therefore map either to environment configuration or to workloads that remain outside Encore. ## How the concepts map | Fly.io | Encore | |---|---| | Fly App | Encore application or environment, depending on how the Fly Apps are divided | | Machine / process group | No direct equivalent; Encore deploys services from the application model | | `http_service` or `services` | Encore API gateway and API endpoints | | Fly Postgres (unmanaged) or Managed Postgres | SQL database, external Postgres, or a migrated database | | Volume | No direct primitive; choose a database, object storage, or externally managed disk | | Secret | Encore secret | | Scheduled Machine | Cron job when the task can be exposed as an API endpoint | | Private network and Flycast | Service-to-service APIs or separately configured private connectivity | | Region and Machine size | Per-environment [infrastructure configuration](https://encore.dev/docs/platform/infrastructure/configuration) | These primitives are declared in application code using [TypeScript](https://encore.dev/docs/ts) or [Go](https://encore.dev/docs/go). A Fly Machine is an independently controllable VM. An Encore service is a logical boundary in the source code, so Machine lifecycle operations, process groups, and per-Machine placement do not translate directly. ## Migrate applications and processes Move HTTP handlers to Encore API endpoints. Existing TypeScript or Go logic can remain behind the endpoints, and an existing router can initially be wrapped with a catch-all handler; see [Migrating an existing system](https://encore.dev/docs/platform/migration/migrate-to-encore). **Encore.ts:** For each process group, identify its trigger and lifecycle. Use [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub) for event consumers and cron for scheduled endpoint invocations. Keep workloads that require SSH access, custom Machine lifecycle control, special kernels, attached GPUs, or an indefinite non-HTTP process outside Encore unless they are redesigned. **Encore.go:** For each process group, identify its trigger and lifecycle. Use [Pub/Sub](https://encore.dev/docs/go/primitives/pubsub) for event consumers and cron for scheduled endpoint invocations. Keep workloads that require SSH access, custom Machine lifecycle control, special kernels, attached GPUs, or an indefinite non-HTTP process outside Encore unless they are redesigned. Do not copy `fly.toml` settings directly. For HTTP services, ports and handlers become Encore API definitions; CPU, memory, regions, and scaling are deployment settings. Fly health checks, concurrency settings, autostop/autostart behavior, and Machine restart policy need to be reconsidered for the target deployment. ## Migrate Postgres, volumes, and secrets The Encore platform does not directly import Fly Postgres (unmanaged) or Fly.io Managed Postgres. Create a target Postgres database and transfer the schema and data, or keep the existing database temporarily as an external dependency. Reachability differs between the two. Fly Postgres (unmanaged) runs as a Fly App that can be given a public IP and a TLS-terminated Postgres service, so an external consumer can connect to it directly, and it remains your operational responsibility. Fly.io Managed Postgres runs inside your Fly.io private network and is not reachable over the public internet, so an external consumer needs WireGuard or `fly mpg proxy`. Preserve the appropriate backup, failover, and monitoring plan until migration is complete. **Encore.ts:** Volumes are local to a server and tied to a region. Encore has no equivalent disk primitive. Move files to [object storage](https://encore.dev/docs/ts/primitives/object-storage), structured data to a database, or keep an external storage system. Applications using multiple volumes must already account for replication or partitioning; preserve those semantics during transfer. **Encore.go:** Volumes are local to a server and tied to a region. Encore has no equivalent disk primitive. Move files to [object storage](https://encore.dev/docs/go/primitives/object-storage), structured data to a database, or keep an external storage system. Applications using multiple volumes must already account for replication or partitioning; preserve those semantics during transfer. Copy sensitive values from Fly secrets into Encore secrets and set them per environment. `fly.toml` environment values that are not secret should become constants or external configuration where appropriate. ## Networking, domains, and coexistence Fly private networking names, Flycast addresses, Anycast services, and WireGuard connectivity do not carry over automatically. Replace internal calls between migrated components with Encore service APIs. For components that remain on Fly.io, establish a reachable, authenticated endpoint or private connection supported by the target cloud; do not assume `.internal` Fly DNS is available outside the Fly organization network. Deploy and verify Encore using its environment URL before changing DNS. Add the hostname as an Encore [custom domain](https://encore.dev/docs/platform/integrations/custom-domains), then update DNS. Geographic routing and data residency may change, so confirm target regions and latency before cutover. ## Migration plan 1. Inventory Fly Apps, process groups, Machines, regions, volumes, Postgres clusters, secrets, and private-network dependencies. 2. Move one stateless HTTP component to Encore and keep its external API stable. 3. Redesign workers and scheduled Machines according to their triggers. 4. Establish connectivity to components that remain on Fly.io. 5. Rehearse database and volume transfers, including the final write freeze or replication cutover. 6. Verify regions, domains, health, and latency before shifting traffic. 7. Retain Fly Machines and volumes until data and rollback requirements are satisfied. For shared PaaS guidance, see [Coming from a PaaS](https://encore.dev/docs/platform/migration/from-paas). Fly.io behavior referenced here is documented in [Machines](https://fly.io/docs/machines/), [scheduled Machines](https://fly.io/docs/machines/flyctl/fly-machine-run/#start-a-machine-on-a-schedule), [Volumes](https://fly.io/docs/volumes/), [Private Networking](https://fly.io/docs/networking/private-networking/), [Fly Postgres (unmanaged)](https://fly.io/docs/postgres/), and [Managed Postgres](https://fly.io/docs/mpg/). --- Source: https://encore.dev/docs/ai-migration Section: Guides # Migrate using AI agent Encore's AI migration skill analyzes your existing backend, builds a dependency-aware migration plan, and converts your code to Encore — one unit at a time, with validation at every step. It works with any source framework: Express, Fastify, Hono, Koa, NestJS, Gin, Echo, Chi, Fiber, net/http, and more. > **Note** > > The skill has been tested with Claude Code but should work with other agents as well. ## Prerequisites Install the Encore skills package in your AI coding tool: ```bash npx skills add encoredev/skills ``` See [Install Encore skills](https://encore.dev/docs/ai-integration#install-encore-skills) for installing them for a specific agent or for all your projects. You'll also need: - The source codebase accessible on your local machine - An Encore project to migrate into (the skill can help create one) - Your source application running locally (optional — enables HTTP comparison validation) ## Starting a migration Create a new Encore app from the "Empty app" template by running: ```bash encore app create ``` From inside your Encore app, open your AI coding tool and ask it to migrate your existing app: ``` Migrate ../path/to/existing/project to Encore.ts by using the encore-migrate skill ``` The skill walks you through four phases: **Discover**, **Plan**, **Migrate**, and **Complete**. ## How it works ### Phase 1 — Discover The AI reads your source codebase and inventories everything: API endpoints, databases, Pub/Sub topics, cron jobs, auth middleware, secrets, and tests. It groups related entities into **migration units** — typically aligned with your existing service boundaries or URL path prefixes — and presents a summary for you to review. You can adjust the groupings before moving on. Split units that are too large, merge ones that are too small, or rename them to match your domain. ### Phase 2 — Plan The AI creates a `migration-plan.md` file and a `migration-plan/` directory in your Encore project. The summary file tracks overall progress and dependency order. Each migration unit gets its own detail file listing every endpoint, database table, and test to migrate. Dependencies determine the order. Secrets and config go first, then databases, auth, leaf services, dependent services, Pub/Sub, and finally cron jobs. ### Phase 3 — Migrate The AI works through one migration unit at a time. For each entity it: 1. **Implements** the Encore equivalent — [API endpoints](https://encore.dev/docs/ts/primitives/defining-apis), [database schemas](https://encore.dev/docs/ts/primitives/databases), [infrastructure declarations](https://encore.dev/docs/ts/primitives/services) 2. **Migrates tests** from the source framework to Encore's [testing patterns](https://encore.dev/docs/ts/develop/testing) 3. **Validates** the result using up to three layers (see [Validation](#validation)) 4. **Updates the plan** files to track progress After completing a unit, it suggests the next one based on the dependency order. You can also pick a different unit or tell it to keep going through multiple units. ### Phase 4 — Complete When all units are done, the AI presents a final summary: what was migrated, what was skipped, and what needs manual attention. It suggests a final test suite run and, if your source system has a frontend, recommends reconnecting it to the new Encore backend using the [Request Client](https://encore.dev/docs/ts/frontend/request-client). ## Full-stack and monorepo support When the source codebase contains frontend code (React, Vue, Angular, Next.js, etc.), the AI identifies it and marks it as out of scope — only backend code is migrated. For full-stack frameworks like **Next.js**, **Remix**, **Nuxt**, **SvelteKit**, and **Astro**, the AI detects server-side routes (e.g., Next.js `pages/api/` or Remix `loader` functions) and asks what you want to do with them: 1. **Migrate all** server-side routes to Encore 2. **Migrate some** — you pick which ones move 3. **Keep all in the frontend framework** — only migrate standalone backend code This is useful when you want an Encore backend but prefer to keep a thin BFF or SSR data-fetching layer in your frontend framework. ## Validation Every entity is validated before it's marked as migrated. The AI uses three layers: **Test migration** — Source tests are converted to Encore's [testing patterns](https://encore.dev/docs/ts/develop/testing) and run. They must pass before the entity is marked as done. **HTTP comparison** — When both systems are running locally, the AI calls the same endpoint on both and compares the HTTP status code and response body structure. This layer is skipped for endpoints with side effects or that require auth credentials the AI can't obtain. **Verification gate** — No entity is marked as `migrated` without concrete evidence from the current session: test output, HTTP comparison results, or your explicit approval to skip. ## Resuming across sessions The migration plan is persisted to files in your Encore project, so you can close your editor and come back later. When you resume, the AI reads `migration-plan.md`, reports the current status, and suggests the next unit to work on. ``` Resume the migration ``` ``` What's left to migrate? ``` --- Source: https://encore.dev/docs/migrate-away Section: Guides # Migrate away from Encore > If you love someone, set them free. _We realize most people read this page before even trying Encore, so we start with a perspective on how you might reason about adopting Encore. Read on to see what tools are available for migrating away._ Picking technologies for your project is an important decision. It's tricky because you don't know what the requirements are going to look like in the future. This uncertainty makes many teams opt for maximum flexibility, often without acknowledging this has a significant negative effect on productivity. When designing Encore, we've leaned on standardization to provide a well-integrated and highly productive development workflow. The design is based on the core team's experience building scalable distributed systems at Spotify and Google, complemented with loads of invaluable input from the developer community. In practise Encore is opinionated only in certain areas which are critical for enabling the static analysis used to create Encore's application model. This is fundamental to how Encore can provide its powerful features, like automatically instrumenting distributed tracing, and provisioning and managing cloud infrastructure. ## Accommodating for your unique requirements Many software projects end up having a few novel requirements, which are highly specific to the problem domain. To accommodate for this, Encore is designed to let you go outside of the standardized Backend Framework when you need to, for example: - You can drop down in abstraction level in the API framework using [raw endpoints](https://encore.dev/docs/ts/primitives/defining-apis#raw-endpoints) - You can use tools like the [Terraform provider](https://encore.dev/docs/platform/integrations/terraform) to integrate infrastructure that is not managed by Encore ## Mitigating risk through Open Source and efficiency We believe that adopting Encore is a low-risk decision for several reasons: - There's no upfront investment needed to get the benefits - Encore apps are normal programs where less than 1% of the code is Encore-specific - All infrastructure and data is in your own cloud - It's simple to integrate with cloud services and systems not natively supported by Encore - Everything you need to develop your application is Open Source, including the [parser](https://github.com/encoredev/encore/tree/main/v2/parser), [compiler](https://github.com/encoredev/encore/tree/main/v2/compiler), [runtime](https://github.com/encoredev/encore/tree/main/runtimes) - Everything you need to self-host your application is [Open Source and documented](https://encore.dev/docs/self-host/build) ## What to expect when migrating away If you want to migrate away, we want to ensure this is as smooth as possible! Here are some of the ways Encore is designed to keep your app portable, with minimized lock-in, and the tools provided to aid in migrating away. ### Code changes Building with Encore doesn't require writing your entire application in an Encore-specific way. Encore applications are normal programs where only 1% of the code is specific to Encore's Open Source Backend Framework. This means that the changes required to stop using the Backend Framework is almost exactly the same work you would have needed to do if you hadn't used Encore in the first place, e.g. writing infrastructure boilerplate. There is no added migration cost. ### Deployment If you are self-hosting your application, then you're already done. If you are using the Encore platform to manage deployments and want to migrate to your own solution, you can use the [self-hosting instructions](https://encore.dev/docs/self-host/build) and Open Source CLI tooling. The `encore build docker` command produces a Docker image, containing the compiled application, using exactly the same code path as Encore's CI system to ensure compatibility. Learn more in the [self-hosting docs](https://encore.dev/docs/self-host/build). ### Tell us what you need We're engineers ourselves and we understand the importance of not being constrained by a single technology. We're working every single day on making it even easier to start, and stop, using Encore. If you have specific concerns, questions, or requirements, we'd love to hear from you! Please reach out on [Discord](https://encore.dev/discord) or [send an email](mailto:hello@encore.dev) with your thoughts. --- Source: https://encore.dev/docs/break-up-monolith Section: Guides # Break a monolith into microservices > Evolving your architecture as needed It's common to want to break out specific functionality into separate services. Perhaps you want to independently scale a specific service, or simply want to structure your codebase in smaller pieces. Encore makes it simple to evolve your system architecture over time, and enables you to deploy your application in multiple different ways without making code changes. ## How to break out a service from a monolith As a (slightly silly) example, let's imagine we have a monolith `hello` with two API endpoints `H1` and `H2`. It looks like this: **Encore.ts:** ```ts -- hello/hello.ts -- import { api } from "encore.dev/api"; export const h1 = api( { method: "GET", path: "/hello/:name", expose: true }, async ({ name }: { name: string }): Promise => { return { message: `Hello, ${name}!` }; } ); export const h2 = api( { method: "GET", path: "/yo/:name", expose: true }, async ({ name }: { name: string }): Promise => { return { message: `Yo, ${name}!` }; } ); interface Response { message: string; } ``` **Encore.go:** ```go package hello import ( "context" ) //encore:api public path=/hello/:name func H1(ctx context.Context, name string) (*Response, error) { msg := "Hello, " + name + "!" return &Response{Message: msg}, nil } //encore:api public path=/yo/:name func H2(ctx context.Context, name string) (*Response, error) { msg := "Yo, " + name + "!" return &Response{Message: msg}, nil } type Response struct { Message string } ``` Now we're going to break out the second endpoint into its own separate service. All we need to do is create a new service directory — a directory with an `encore.service.ts` in Encore.ts, a Go package in Encore.go — call it `yo`, and move the endpoint into it. Like so: **Encore.ts:** ```ts -- yo/encore.service.ts -- import { Service } from "encore.dev/service"; export default new Service("yo"); -- yo/yo.ts -- import { api } from "encore.dev/api"; export const h2 = api( { method: "GET", path: "/yo/:name", expose: true }, async ({ name }: { name: string }): Promise => { return { message: `Yo, ${name}!` }; } ); interface Response { message: string; } ``` **Encore.go:** ```go package yo import ( "context" ) //encore:api public path=/yo/:name func H2(ctx context.Context, name string) (*Response, error) { msg := "Yo, " + name + "!" return &Response{Message: msg}, nil } type Response struct { Message string } ``` On disk we now have: **Encore.ts:** ``` /my-app ├── encore.app // ... and other top-level project files │ ├── hello // hello service (a directory) │ ├── encore.service.ts // service definition │ └── hello.ts // hello service code │ └── yo // yo service (a directory) ├── encore.service.ts // service definition └── yo.ts // yo service code ``` **Encore.go:** ``` /my-app ├── encore.app // ... and other top-level project files │ ├── hello // hello service (a Go package) │   └── hello.go // hello service code │ └── yo // yo service (a Go package) └── yo.go // yo service code ``` Encore now understands these are separate services, and when you run your app you'll see that the [Service Catalog](https://encore.dev/docs/service-catalog) has been automatically updated accordingly. ![Service Catalog - Microservices](https://encore.dev/assets/docs/microservices-service-catalog.png) As well as the [architecture diagram](https://encore.dev/docs/architecture). ![Architecture diagram - Microservices](https://encore.dev/assets/docs/microservices-flow.png) ## Sharing databases between services (or not) Deciding whether to share a database between multiple services depends on your specific situation. Encore supports both options. Learn more in the [database documentation](https://encore.dev/docs/go/primitives/share-db-between-services). --- Source: https://encore.dev/docs/ts/develop/integrations/better-auth Section: Guides (Encore.ts) # Better Auth [Better Auth](https://www.better-auth.com) is a TypeScript authentication library that supports email/password, OAuth, two-factor, magic links, and sessions. This guide shows how to use it with Encore's [database provisioning](https://encore.dev/docs/ts/primitives/databases) and [secrets management](https://encore.dev/docs/ts/primitives/secrets). To get started quickly, create a new app from the example: ```shell $ encore app create --example=ts/betterauth ``` Or follow the steps below to add Better Auth to an existing Encore app. > **Note** > > If you haven't installed Encore yet, see the [installation guide](https://encore.dev/docs/quick-start) first. ## Install ```shell $ npm install better-auth pg ``` ## Set up the database Better Auth needs a database for users and sessions. Encore [provisions and manages databases](https://encore.dev/docs/ts/primitives/databases) for you automatically, just define it in code: ```ts -- db.ts -- import { SQLDatabase } from "encore.dev/storage/sqldb"; export const db = new SQLDatabase("auth", { migrations: "./migrations", }); ``` > **Note** > > Locally, Encore starts a PostgreSQL instance automatically when you run `encore run`. You'll need [Docker](https://docker.com/get-started/) running for the local database. ## Configure Better Auth Create the Better Auth instance using Encore's database and secrets: ```ts -- auth.ts -- import { betterAuth } from "better-auth"; import { Pool } from "pg"; import { secret } from "encore.dev/config"; import { db } from "./db"; const authSecret = secret("AuthSecret"); const pool = new Pool({ connectionString: db.connectionString, }); export const auth = betterAuth({ secret: authSecret(), basePath: "/auth", database: pool, trustedOrigins: ["http://localhost:4000"], emailAndPassword: { enabled: true, }, socialProviders: { github: { clientId: secret("GithubClientId")(), clientSecret: secret("GithubClientSecret")(), }, }, }); ``` Set the secrets using the Encore CLI: ```shell $ encore secret set --type dev,local,pr,production AuthSecret $ encore secret set --type dev,local,pr,production GithubClientId $ encore secret set --type dev,local,pr,production GithubClientSecret ``` > **Note** > > **Tip:** Generate a strong auth secret with `openssl rand -base64 32` and paste it when prompted for `AuthSecret`. > **Note** > > Locally, secrets are stored on your machine and injected when you run `encore run`. No `.env` files needed. ## Connect to Encore's auth handler Wire Better Auth into Encore's [authentication system](https://encore.dev/docs/ts/develop/auth) so you can use `auth: true` on any API endpoint: ```ts -- handler.ts -- import { APIError, Gateway } from "encore.dev/api"; import { authHandler } from "encore.dev/auth"; import { Header } from "encore.dev/api"; import { auth } from "./auth"; interface AuthParams { authorization: Header<"Authorization">; } interface AuthData { userID: string; } const handler = authHandler( async (params) => { const session = await auth.api.getSession({ headers: new Headers({ authorization: params.authorization, }), }); if (!session) { throw APIError.unauthenticated("invalid session"); } return { userID: session.user.id }; } ); export const gateway = new Gateway({ authHandler: handler }); ``` ## Expose Better Auth routes Better Auth needs HTTP routes for sign-in, sign-up, and OAuth callbacks. Expose these using a [raw endpoint](https://encore.dev/docs/ts/primitives/raw-endpoints): ```ts -- routes.ts -- import { api } from "encore.dev/api"; import { auth } from "./auth"; // Better Auth expects a Web Request, but Encore raw endpoints receive // a Node.js IncomingMessage. We convert between the two formats. export const authRoutes = api.raw( { expose: true, path: "/auth/*path", method: "*" }, async (req, res) => { // Read the request body const chunks: Buffer[] = []; for await (const chunk of req) { chunks.push(chunk); } const body = Buffer.concat(chunks); // Build a Web Request from the Node.js request const headers = new Headers(); for (const [key, value] of Object.entries(req.headers)) { if (value) headers.append(key, Array.isArray(value) ? value.join(", ") : value); } const url = `http://${req.headers.host}${req.url}`; const webReq = new Request(url, { method: req.method, headers, body: ["GET", "HEAD"].includes(req.method || "") ? undefined : body, }); // Pass to Better Auth and forward the response const response = await auth.handler(webReq); response.headers.forEach((value, key) => { res.setHeader(key, value); }); res.writeHead(response.status); res.end(await response.text()); } ); ``` ## Use in your endpoints Any endpoint with `auth: true` will now require a valid Better Auth session: ```ts import { api } from "encore.dev/api"; import { getAuthData } from "~encore/auth"; export const getProfile = api( { auth: true, expose: true, method: "GET", path: "/profile" }, async (): Promise<{ userID: string }> => { const data = getAuthData()!; return { userID: data.userID }; } ); ``` ## Deploy When you deploy, Encore automatically provisions and manages the infrastructure your app needs: - **Database** provisioned as Cloud SQL on GCP or RDS on AWS. Migrations run automatically on deploy. - **Secrets** encrypted per environment (preview, staging, production), never shared between them. - **Networking** including TLS, load balancing, and DNS. ### Self-hosting Build a Docker image and deploy anywhere. The [infra config file](https://encore.dev/docs/self-host/configure-infra) tells the app how to reach your infrastructure: ```shell $ encore build docker --config infra-config.json my-app:latest ``` See the [self-hosting docs](https://encore.dev/docs/self-host/build) for more details. ### The Encore platform Deploy your application to a free staging environment in Encore's development cloud: ```shell $ git add -A . $ git commit -m 'Deploy' $ git push encore ``` You can also connect your own AWS or GCP account and Encore will automatically provision databases, run migrations, and manage secrets in your cloud. See [Connect your cloud account](https://encore.dev/docs/platform/deploy/own-cloud) for details. ## Related resources - [Encore authentication docs](https://encore.dev/docs/ts/develop/auth) - [Better Auth documentation](https://www.better-auth.com/docs) - [Encore databases](https://encore.dev/docs/ts/primitives/databases) - [Encore secrets](https://encore.dev/docs/ts/primitives/secrets) --- Source: https://encore.dev/docs/ts/develop/integrations/polar Section: Guides (Encore.ts) # Polar [Polar](https://polar.sh) handles payments, subscriptions, and license keys as your Merchant of Record. This guide shows how to integrate Polar with an Encore application. To get started quickly, create a new app from the example: ```shell $ encore app create --example=ts/polar ``` Or follow the steps below to add Polar to an existing Encore app. > **Note** > > If you haven't installed Encore yet, see the [installation guide](https://encore.dev/docs/quick-start) first. ## Install the SDK ```shell $ npm install @polar-sh/sdk ``` ## Polar setup Before writing code, you'll need to set up a few things in the [Polar dashboard](https://sandbox.polar.sh) (use the sandbox for development): 1. **Create an access token.** Go to Settings > [Developers > Personal Access Tokens](https://sandbox.polar.sh/settings/developers/pat) and create a new token. 2. **Create a product.** Go to [Products](https://sandbox.polar.sh/products) and create at least one product. Copy its **product ID**, you'll need it to create checkout sessions. 3. **Set up a webhook** (optional for local dev). Go to Settings > [Webhooks](https://sandbox.polar.sh/settings/webhooks) and point it to your API URL followed by `/webhooks/polar`. For local development, use a tunnel like [ngrok](https://ngrok.com) to expose your local server. See the [Polar documentation](https://docs.polar.sh) for more details on products, pricing, and webhooks. ## Set your secrets Store your Polar credentials as [Encore secrets](https://encore.dev/docs/ts/primitives/secrets): ```shell $ encore secret set --type dev,local,pr,production PolarAccessToken ``` > **Note** > > Locally, secrets are stored on your machine and injected when you run `encore run`. No `.env` files needed. ## Initialize the client Create a file to configure the Polar SDK. Use Encore's [`secret()`](https://encore.dev/docs/ts/primitives/secrets) function to access the token. Use `sandbox` for development and `production` when deployed: ```ts -- polar.ts -- import { Polar } from "@polar-sh/sdk"; import { secret } from "encore.dev/config"; const polarAccessToken = secret("PolarAccessToken"); const server = process.env.ENCORE_ENVIRONMENT === "production" ? "production" : "sandbox"; export const polar = new Polar({ accessToken: polarAccessToken(), server, }); ``` ## Create a checkout Use the Polar SDK to create checkout sessions for your products: ```ts -- checkout.ts -- import { api } from "encore.dev/api"; import { polar } from "./polar"; import { getAuthData } from "~encore/auth"; interface CreateCheckoutRequest { productId: string; } interface CreateCheckoutResponse { checkoutUrl: string; } export const createCheckout = api( { auth: true, expose: true, method: "POST", path: "/checkout" }, async (req: CreateCheckoutRequest): Promise => { const authData = getAuthData()!; const baseUrl = process.env.ENCORE_API_URL || "http://localhost:4000"; const session = await polar.checkouts.create({ products: [req.productId], customerEmail: authData.email, successUrl: `${baseUrl}/?success=true`, }); return { checkoutUrl: session.url || "" }; } ); ``` ## Handle webhooks Create a [raw endpoint](https://encore.dev/docs/ts/primitives/raw-endpoints) to receive webhook events from Polar: ```ts -- webhooks.ts -- import { api } from "encore.dev/api"; import log from "encore.dev/log"; export const handleWebhook = api.raw( { expose: true, path: "/webhooks/polar", method: "POST" }, async (req, res) => { const chunks: Buffer[] = []; for await (const chunk of req) { chunks.push(chunk); } const event = JSON.parse(Buffer.concat(chunks).toString()); log.info("Received Polar webhook", { type: event.type }); switch (event.type) { case "subscription.active": // Grant access to your product break; case "subscription.canceled": // Revoke access break; case "order.paid": // Fulfill the order break; } res.writeHead(200); res.end(); } ); ``` Register your webhook URL in the [Polar dashboard](https://sandbox.polar.sh/settings/webhooks) under Settings > Webhooks. Use your Encore API URL followed by `/webhooks/polar`. Enable the events you want to handle (e.g. `subscription.active`, `subscription.canceled`, `order.paid`). ## Deploy When you deploy, Encore automatically provisions and manages the infrastructure your app needs: - **Secrets** encrypted per environment (preview, staging, production), never shared between them. - **Databases** provisioned as Cloud SQL on GCP or RDS on AWS. - **Networking** including TLS, load balancing, and DNS. ### Self-hosting Build a Docker image and deploy anywhere: ```shell $ encore build docker my-app:latest ``` See the [self-hosting docs](https://encore.dev/docs/self-host/build) for more details. ### The Encore platform Deploy your application to a free staging environment in Encore's development cloud: ```shell $ git push encore main ``` You can also connect your own AWS or GCP account and Encore will automatically provision the infrastructure and manage secrets in your cloud. See [Connect your cloud account](https://encore.dev/docs/platform/deploy/own-cloud) for details. ## Related resources - [Polar + Encore example app](https://github.com/encoredev/examples/tree/main/ts/polar) - [Polar documentation](https://docs.polar.sh) - [Polar sandbox dashboard](https://sandbox.polar.sh) - [Encore secrets](https://encore.dev/docs/ts/primitives/secrets) - [Raw endpoints](https://encore.dev/docs/ts/primitives/raw-endpoints) --- Source: https://encore.dev/docs/ts/develop/integrations/resend Section: Guides (Encore.ts) # Resend [Resend](https://resend.com) provides transactional email with high deliverability and React Email templates. This guide shows how to use it with Encore's [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub) and [secrets management](https://encore.dev/docs/ts/primitives/secrets). To get started quickly, create a new app from the example: ```shell $ encore app create --example=ts/resend ``` Or follow the steps below to add Resend to an existing Encore app. > **Note** > > If you haven't installed Encore yet, see the [installation guide](https://encore.dev/docs/quick-start) first. ## Install the SDK ```shell $ npm install resend ``` If you want to use React Email templates: ```shell $ npm install resend @react-email/components ``` ## Resend setup Before writing code, you'll need to configure a few things in the [Resend dashboard](https://resend.com): 1. **Create an API key.** Go to [API Keys](https://resend.com/api-keys) and create a new key. 2. **Verify a domain** (optional for testing). Go to [Domains](https://resend.com/domains) and add your sending domain. Until you verify a domain, you can use `onboarding@resend.dev` as the `from` address for testing. See the [Resend documentation](https://resend.com/docs) for more details on domain verification and sending limits. ## Set your API key Store your Resend API key as an [Encore secret](https://encore.dev/docs/ts/primitives/secrets): ```shell $ encore secret set --type dev,local,pr,production ResendAPIKey ``` > **Note** > > Locally, secrets are stored on your machine and injected when you run `encore run`. No `.env` files needed. ## Initialize the client ```ts -- resend.ts -- import { Resend } from "resend"; import { secret } from "encore.dev/config"; const resendApiKey = secret("ResendAPIKey"); export const resend = new Resend(resendApiKey()); ``` ## Send an email Use the Resend SDK in an Encore API endpoint: ```ts -- send.ts -- import { api } from "encore.dev/api"; import { resend } from "./resend"; interface SendEmailRequest { to: string; subject: string; html: string; } interface SendEmailResponse { id: string; } export const sendEmail = api( { expose: true, method: "POST", path: "/email/send" }, async (req: SendEmailRequest): Promise => { const { data, error } = await resend.emails.send({ from: "Your App ", to: req.to, subject: req.subject, html: req.html, }); if (error) { throw new Error(`Failed to send email: ${error.message}`); } return { id: data!.id }; } ); ``` > **Note** > > The `from` address must use a domain you've verified in [Resend](https://resend.com/domains). For testing, you can use `onboarding@resend.dev` which works with any API key. ## Async delivery with Pub/Sub For better performance, send emails asynchronously using Encore's [Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub). This keeps your API endpoints fast and handles retries automatically: ```ts -- topic.ts -- import { Topic, Subscription } from "encore.dev/pubsub"; import { resend } from "./resend"; interface EmailEvent { to: string; subject: string; html: string; } export const emailTopic = new Topic("email-send", { deliveryGuarantee: "at-least-once", }); const _ = new Subscription(emailTopic, "send-via-resend", { handler: async (event) => { const { error } = await resend.emails.send({ from: "Your App ", to: event.to, subject: event.subject, html: event.html, }); if (error) { throw new Error(error.message); } }, }); ``` Then publish from any endpoint: ```ts import { emailTopic } from "./topic"; // Inside any API endpoint await emailTopic.publish({ to: "user@example.com", subject: "Welcome!", html: "

Thanks for signing up.

", }); ``` > **Note** > > Locally, Pub/Sub runs in-process so messages are delivered immediately, making it easy to test and debug. ## Deploy When you deploy, Encore automatically provisions and manages the infrastructure your app needs: - **Secrets** encrypted per environment (preview, staging, production), never shared between them. - **Pub/Sub** provisioned as GCP Pub/Sub or SQS/SNS on AWS, with automatic retries and dead-letter queues. - **Networking** including TLS, load balancing, and DNS. ### Self-hosting Build a Docker image and deploy anywhere. The [infra config file](https://encore.dev/docs/self-host/configure-infra) tells the app how to reach your infrastructure: ```shell $ encore build docker --config infra-config.json my-app:latest ``` See the [self-hosting docs](https://encore.dev/docs/self-host/build) for more details. ### The Encore platform Deploy your application to a free staging environment in Encore's development cloud: ```shell $ git add -A . $ git commit -m 'Deploy' $ git push encore ``` You can also connect your own AWS or GCP account and Encore will automatically provision Pub/Sub topics, manage secrets, and handle networking in your cloud. See [Connect your cloud account](https://encore.dev/docs/platform/deploy/own-cloud) for details. ## Related resources - [Resend + Encore example app](https://github.com/encoredev/examples/tree/main/ts/resend) - [Resend documentation](https://resend.com/docs) - [Encore Pub/Sub](https://encore.dev/docs/ts/primitives/pubsub) - [Encore secrets](https://encore.dev/docs/ts/primitives/secrets) --- Source: https://encore.dev/docs/ts/migration/express-migration Section: Guides (Encore.ts) # Migrating from Express.js If you have an existing app using [Express.js](https://expressjs.com/) and want to migrate it to Encore.ts, this guide is for you. This guide can also serve as a comparison between the two frameworks. ## Why migrate to Encore.ts? Express.js is a great choice for building simple APIs, but as your application grows you will likely run into limitations. There is a large community around Express.js, providing many plugins and middleware to work around these limitations. However relying heavily on plugins can make it hard to find the right tools for your use case. It also means that you will need to maintain a lot of dependencies. Encore.ts is a framework that aims to make it easier to build robust and type-safe backends with TypeScript. Encore.ts has 0 npm dependencies, is built with performance in mind, and has a lot of built-in features for building production ready backends. You can deploy an Encore.ts app to any hosting service that accepts Docker containers, or use [the Encore platform](https://encore.dev/docs/platform/introduction) to fully automate your DevOps and infrastructure. ### Performance Unlike a lot of other Node.js frameworks, Encore.ts is not built on top of Express.js. Instead, Encore.ts has its own high-performance runtime, with a multi-threaded, asynchronous event loop written in Rust. The Encore Runtime handles all I/O like accepting and processing incoming HTTP requests. This runs as a completely independent event loop that utilizes as many threads as the underlying hardware supports. The result of this is that Encore.ts performs **9x faster** than Express.js. Learn more about the [Encore.ts Runtime](https://encore.dev/blog/event-loops). ### Built-in benefits When using Encore.ts you get a lot of built-in features without having to install any additional dependencies: | Built-in benefits | | | | :----------------------------------------------------- | :--------------------------------------------------: | -------------------------------------------------------------: | | [Pub/Sub integrations](https://encore.dev/docs/ts/primitives/pubsub) | [Type-safe API schemas](https://encore.dev/docs/ts/primitives/defining-apis) | [API Client generation](https://encore.dev/docs/ts/cli/client-generation) | | [Secrets management](https://encore.dev/docs/ts/primitives/secrets) | [CORS handling](https://encore.dev/docs/cors) | [Local Development Dashboard](https://encore.dev/docs/dev-dash) | | [Database integrations](https://encore.dev/docs/ts/primitives/databases) | [Architecture Diagrams](https://encore.dev/docs/architecture) | [Service Catalog](https://encore.dev/docs/service-catalog) | | [Request validation](https://encore.dev/blog/event-loops) | [Cron Jobs](https://encore.dev/docs/ts/primitives/cron-jobs) | [Local tracing](https://encore.dev/docs/tracing) | ## Migration guide Below we've outlined two main strategies you can use to migrate your existing Express.js application to Encore.ts. Pick the strategy that best suits your situation and application. **Related example:** [encoredev/examples/ts/expressjs-migration](https://github.com/encoredev/examples/tree/main/ts/expressjs-migration) Code examples for migrating an Express.js app to Encore.ts Create it locally with `encore app create --example=ts/expressjs-migration`. ### Forklift migration (quick start) When you quickly want to migrate to Encore.ts and don't need all the functionality to begin with, you can use a forklift migration strategy. This approach moves the entire application over to Encore.ts in one shot, by wrapping your existing HTTP router in a catch-all handler. **Approach benefits** - You can get your application up and running with Encore.ts quickly and start moving features over to Encore.ts while the rest of the application is still untouched. - You will see a partial performance boost right away because the HTTP layer is now running on the Encore Rust runtime. But to get the full performance benefits, you will need to start using Encore's [API declarations](https://encore.dev/docs/ts/primitives/defining-apis) and [infrastructure declarations](https://encore.dev/docs/ts#the-infrastructure-primitives). **Approach drawbacks** - Because all requests will be proxied through the catch-all handler, you will not be able to get all the benefits from the [distributed tracing](https://encore.dev/docs/tracing), which rely on the [Encore application model](https://encore.dev/docs/understanding-encore). - The [architecture diagram](https://encore.dev/docs/architecture) and the [Service Catalog](https://encore.dev/docs/service-catalog) will not be able to show you the full picture of your application until you start moving services and APIs over to Encore.ts. - You will not be able to use the [API Client generation](https://encore.dev/docs/ts/cli/client-generation) feature until you start defining APIs in Encore.ts. #### 1. Install Encore If this is the first time you're using Encore, you first need to install the CLI that runs the local development environment. Use the appropriate command for your system: - **macOS:** `brew install encoredev/tap/encore` - **Linux:** `curl -L https://encore.dev/install.sh | bash` - **Windows:** `iwr https://encore.dev/install.ps1 | iex` [Installation docs](https://encore.dev/docs/quick-start) #### 2. Add Encore.ts to your project ```bash npm i encore.dev ``` #### 3. Initialize an Encore app Inside your project directory, run the following command to create an Encore app: ```bash encore app init ``` This will create an `encore.app` file in the root of your project. #### 4. Configure your tsconfig.json To the `tsconfig.json` file in the root of your project, add the following: ```json -- tsconfig.json -- { "compilerOptions": { "paths": { "~encore/*": [ "./encore.gen/*" ] } } } ``` When Encore.ts is parsing your code it will specifically look for `~encore/*` imports. #### 5. Define an Encore.ts service When running an app using Encore.ts you need at least one [Encore service](https://encore.dev/docs/ts/primitives/services). Apart from that, Encore.ts in not opinionated in how you structure your code, you are free to go with a monolith or microservice approach. Learn more in our [App Structure docs](https://encore.dev/docs/app-structure). In the root of your App, add a file named `encore.service.ts`. The file must export a service instance, by calling `new Service`, imported from `encore.dev/service`: ```ts -- encore.service.ts -- import {Service} from "encore.dev/service"; export default new Service("my-service"); ``` Encore will consider this directory and all its subdirectories as part of the service. #### 6. Create a catch-all handler for your HTTP router Now let's mount your existing app router under a [Raw endpoint](https://encore.dev/docs/ts/primitives/raw-endpoints), which is an Encore API endpoint type that gives you access to the underlying HTTP request. Here's a basic code example: ```typescript import { api, RawRequest, RawResponse } from "encore.dev/api"; import express, { request, response } from "express"; Object.setPrototypeOf(request, RawRequest.prototype); Object.setPrototypeOf(response, RawResponse.prototype); const app = express(); app.get('/foo', (req: any, res) => { res.send('Hello World!') }) export const expressApp = api.raw( { expose: true, method: "*", path: "/!rest" }, app, ); ``` By mounting your existing app router in this way, it will work as a catch-all handler for all HTTP requests and responses. #### 7. Run you app locally You will now be able to run your Express.js app locally using the `encore run` command. #### Next steps: Incrementally move over Encore.ts to get all the benefits You can now gradually break out specific endpoints using the Encore's [API declarations](#apis) and introduce infrastructure declarations for databases and cron jobs etc. This will let Encore.ts understand your application and unlock all Encore.ts features. See the [Feature-by-feature migration](#feature-by-feature-migration) section for more details. You will eventually be able to remove Express.js as a dependency and run your app entirely on Encore.ts. You can also [join Discord](https://encore.dev/discord) to ask questions and meet fellow Encore developers. #### Forklift example _[Interactive editor showing the finished `expressForklift` app — open this page in a browser to see it.]_ ### Full migration This approach aims to fully replace your applications dependency on Express.js with Encore.ts, unlocking all the features and performance of Encore.ts. Below are two examples that you can use to identify the refactoring you will need to do. In the next section you will find a [Feature-by-feature migration](#feature-by-feature-migration) guide to help you with the refactoring details. **Approach benefits** - Get all the advantages of Encore.ts, like [distributed tracing](https://encore.dev/docs/tracing) and [architecture diagrams](https://encore.dev/docs/architecture), which rely on the [Encore application model](https://encore.dev/docs/understanding-encore). - Get the [full performance benefit](https://encore.dev/blog/event-loops) of Encore.ts - **9x faster** than Express.js. **Approach drawbacks** - This approach may require more time and effort up front compared to the [Incremental migration strategy](#incremental-migration-strategy). #### App comparison Here is a side-by-side comparison of an Express.js app and an Encore.ts app. The examples show how to create APIs, handle request validation, error handling, serving static files, and rendering templates. **Express.js** _[Interactive editor showing the finished `expressVsEncore` app — open this page in a browser to see it.]_ **Encore.ts** _[Interactive editor showing the finished `encoreVsExpress` app — open this page in a browser to see it.]_ ## Feature-by-feature migration Check out our [Express.js compared to Encore.ts example](https://github.com/encoredev/examples/tree/main/ts/expressjs-migration) on GitHub for all of the code snippets in this feature comparison. ### APIs With Express.js, you create APIs using the `app.get`, `app.post`, `app.put`, `app.delete` functions. These functions take a path and a callback function. You then use the `req` and `res` objects to handle the request and response. With Encore.ts, you create APIs using the `api` function. This function takes an options object and a callback function. The main difference compared to Express.js is that Encore.ts is type-safe, meaning you define the request and response schemas in the callback function. You then return an object matching the response schema. In case you need to operate at a lower abstraction level, Encore supports defining raw endpoints that let you access the underlying HTTP request. Learn more in the [API Schemas docs](https://encore.dev/docs/ts/primitives/defining-apis#api-schemas). **Express.js** ```typescript import express, {Request, Response} from "express"; const app: Express = express(); // GET request with dynamic path parameter app.get("/hello/:name", (req: Request, res: Response) => { const msg = `Hello ${req.params.name}!`; res.json({message: msg}); }) // GET request with query string parameter app.get("/hello", (req: Request, res: Response) => { const msg = `Hello ${req.query.name}!`; res.json({message: msg}); }); // POST request example with JSON body app.post("/order", (req: Request, res: Response) => { const price = req.body.price; const orderId = req.body.orderId; // Handle order logic res.json({message: "Order has been placed"}); }); ``` **Encore.ts** ```typescript import {api, Query} from "encore.dev/api"; // Dynamic path parameter :name export const dynamicPathParamExample = api( {expose: true, method: "GET", path: "/hello/:name"}, async ({name}: { name: string }): Promise<{ message: string }> => { const msg = `Hello ${name}!`; return {message: msg}; }, ); interface RequestParams { // Encore will now automatically parse the query string parameter name?: Query; } // Query string parameter ?name export const queryStringExample = api( {expose: true, method: "GET", path: "/hello"}, async ({name}: RequestParams): Promise<{ message: string }> => { const msg = `Hello ${name}!`; return {message: msg}; }, ); interface OrderRequest { price: string; orderId: number; } // POST request example with JSON body export const order = api( {expose: true, method: "POST", path: "/order"}, async ({price, orderId}: OrderRequest): Promise<{ message: string }> => { // Handle order logic console.log(price, orderId) return {message: "Order has been placed"}; }, ); // Raw endpoint export const myRawEndpoint = api.raw( {expose: true, path: "/raw", method: "GET"}, async (req, resp) => { resp.writeHead(200, {"Content-Type": "text/plain"}); resp.end("Hello, raw world!"); }, ); ``` ### Microservices communication Express.js does not have built-in support for creating microservices or for service-to-service communication. You will most likely use `fetch` or something equivalent to call another service. With Encore.ts, calling another service is just like calling a local function, with complete type-safety. Under the hood, Encore.ts will translate this function call into an actual service-to-service HTTP call, resulting in trace data being generated for each call. Learn more in the [service communication guidance](https://encore.dev/docs/app-structure#choose-how-services-communicate). **Express.js** ```typescript import express, {Request, Response} from "express"; const app: Express = express(); app.get("/save-post", async (req: Request, res: Response) => { try { // Calling another service using fetch const resp = await fetch("https://another-service/posts", { method: "POST", headers: {"Content-Type": "application/json"}, body: JSON.stringify({ title: req.query.title, content: req.query.content, }), }); res.json(await resp.json()); } catch (e) { res.status(500).json({error: "Could not save post"}); } }); ``` **Encore.ts** ```typescript import {api} from "encore.dev/api"; import {anotherService} from "~encore/clients"; export const microserviceCommunication = api( {expose: true, method: "GET", path: "/call"}, async (): Promise<{ message: string }> => { // Calling the foo endpoint in anotherService const fooResponse = await anotherService.foo(); const msg = `Data from another service ${fooResponse.data}!`; return {message: msg}; }, ); ``` ### Authentication In Express.js you can create a middleware function that checks if the user is authenticated. You can then use this middleware function in your routes to protect them. You will have to specify the middleware function for each route that requires authentication. With Encore.ts, when an API is defined with `auth: true`, you must define an authentication handler in your application. The authentication handler is responsible for inspecting incoming requests to determine what user is authenticated. The authentication handler is defined similarly to API endpoints, using the `authHandler` function imported from `encore.dev/auth`. Like API endpoints, the authentication handler defines what request information it's interested in, in the form of HTTP headers, query strings, or cookies. If a request has been successfully authenticated, the API Gateway forwards the authentication data to the target endpoint. The endpoint can query the available auth data from the `getAuthData` function, available from the `~encore/auth` module. Learn more in our [Auth Handler docs](https://encore.dev/docs/ts/develop/auth) **Express.js** ```typescript import express, {NextFunction, Request, Response} from "express"; const app: Express = express(); // Auth middleware function authMiddleware(req: Request, res: Response, next: NextFunction) { // TODO: Validate up auth token and verify that this is an authenticated user const isInvalidUser = req.headers["authorization"] === undefined; if (isInvalidUser) { res.status(401).json({error: "invalid request"}); } else { next(); } } // Endpoint that requires auth app.get("/dashboard", authMiddleware, (_, res: Response) => { res.json({message: "Secret dashboard message"}); }); ``` **Encore.ts** ```typescript import { api, APIError, Gateway, Header } from "encore.dev/api"; import { authHandler } from "encore.dev/auth"; import { getAuthData } from "~encore/auth"; interface AuthParams { authorization: Header<"Authorization">; } // The function passed to authHandler will be called for all incoming API call that requires authentication. export const myAuthHandler = authHandler( async (params: AuthParams): Promise<{ userID: string }> => { // TODO: Validate up auth token and verify that this is an authenticated user const isInvalidUser = params.authorization === undefined; if (isInvalidUser) { throw APIError.unauthenticated("Invalid user ID"); } return { userID: "user123" }; }, ); export const gateway = new Gateway({ authHandler: myAuthHandler }); // Auth endpoint example export const dashboardEndpoint = api( // Setting auth to true will require the user to be authenticated { auth: true, method: "GET", path: "/dashboard" }, async (): Promise<{ message: string; userID: string }> => { return { message: "Secret dashboard message", userID: getAuthData()!.userID, }; }, ); ``` ### Request validation Express.js does not have built-in request validation. You have to use a library like [Zod](https://github.com/colinhacks/zod). With Encore.ts, request validation for headers, query params and body is. You supply a schema for the request object and in the request payload does not match the schema the API will return a 400 error. Learn more in the [API Schemas docs](https://encore.dev/docs/ts/primitives/defining-apis#api-schemas). **Express.js** ```typescript import express, {NextFunction, Request, Response} from "express"; import {z, ZodError} from "zod"; const app: Express = express(); // Request validation middleware function validateData(schemas: { body: z.ZodObject; query: z.ZodObject; headers: z.ZodObject; }) { return (req: Request, res: Response, next: NextFunction) => { try { // Validate headers schemas.headers.parse(req.headers); // Validate request body schemas.body.parse(req.body); // Validate query params schemas.query.parse(req.query); next(); } catch (error) { if (error instanceof ZodError) { const errorMessages = error.errors.map((issue: any) => ({ message: `${issue.path.join(".")} is ${issue.message}`, })); res.status(400).json({error: "Invalid data", details: errorMessages}); } else { res.status(500).json({error: "Internal Server Error"}); } } }; } // Request body validation schemas const bodySchema = z.object({ someKey: z.string().optional(), someOtherKey: z.number().optional(), requiredKey: z.array(z.number()), nullableKey: z.number().nullable().optional(), multipleTypesKey: z.union([z.boolean(), z.number()]).optional(), enumKey: z.enum(["John", "Foo"]).optional(), }); // Query string validation schemas const queryStringSchema = z.object({ name: z.string().optional(), }); // Headers validation schemas const headersSchema = z.object({ "x-foo": z.string().optional(), }); // Request validation example using Zod app.post( "/validate", validateData({ headers: headersSchema, body: bodySchema, query: queryStringSchema, }), (_: Request, res: Response) => { res.json({message: "Validation succeeded"}); }, ); ``` **Encore.ts** ```typescript import {api, Header, Query} from "encore.dev/api"; enum EnumType { FOO = "foo", BAR = "bar", } // Encore.ts automatically validates the request schema and returns and error // if the request does not match the schema. interface RequestSchema { foo: Header<"x-foo">; name?: Query; someKey?: string; someOtherKey?: number; requiredKey: number[]; nullableKey?: number | null; multipleTypesKey?: boolean | number; enumKey?: EnumType; } // Validate a request export const schema = api( {expose: true, method: "POST", path: "/validate"}, (data: RequestSchema): { message: string } => { console.log(data); return {message: "Validation succeeded"}; }, ); ``` ### Error handling In Express.js you either throw an error (which results in a 500 response) or use the `status` function to set the status code of the response. In Encore.ts throwing an error will result in a 500 response. You can also use the `APIError` class to return specific error codes. Learn more in our [API Errors docs](https://encore.dev/docs/ts/primitives/errors). **Express.js** ```typescript import express, {Request, Response} from "express"; const app: Express = express(); // Default error handler app.get("/broken", (req, res) => { throw new Error("BROKEN"); // This will result in a 500 error }); // Returning specific error code app.get("/get-user", (req: Request, res: Response) => { const id = req.query.id || ""; if (id.length !== 3) { res.status(400).json({error: "invalid id format"}); } // TODO: Fetch something from the DB res.json({user: "Simon"}); }); ``` **Encore.ts** ```typescript import {api, APIError} from "encore.dev/api"; // Default error handler // Default error handler export const broken = api( {expose: true, method: "GET", path: "/broken"}, async (): Promise => { throw new Error("This is a broken endpoint"); // This will result in a 500 error }, ); // Returning specific error code export const brokenWithErrorCode = api( {expose: true, method: "GET", path: "/broken/:id"}, async ({id}: { id: string }): Promise<{ user: string }> => { if (id.length !== 3) { throw APIError.invalidArgument("invalid id format"); } // TODO: Fetch something from the DB return {user: "Simon"}; }, ); ``` ### Serving static files Express.js has a built-in middleware function to serve static files. You can use the `express.static` function to serve files from a specific directory. Encore.ts also has built-in support for static file serving with the `api.static` method. The files are served directly from the Encore.ts Rust Runtime. This means that zero JavaScript code is executed to serve the files, freeing up the Node.js runtime to focus on executing business logic. This dramatically speeds up both the static file serving, as well as improving the latency of your API endpoints. Learn more in our [Static Assets docs](https://encore.dev/docs/ts/primitives/static-assets). **Express.js** ```typescript import express from "express"; const app: Express = express(); app.use("/assets", express.static("assets")); ``` **Encore.ts** ```typescript import { api } from "encore.dev/api"; export const assets = api.static( { expose: true, path: "/assets/*path", dir: "./assets" }, ); ``` ### Template rendering Express.js has a built-in support for rendering templates. With Encore.ts you can use the `api.raw` function to serve HTML templates, in this example we are using Handlebars.js but you can use whichever templating engine you prefer. Learn more in our [Raw Endpoints docs](https://encore.dev/docs/ts/primitives/raw-endpoints) **Express.js** ```typescript import express, {Request, Response} from "express"; const app: Express = express(); app.set("view engine", "pug"); // Set view engine to Pug // Template engine example. This will render the index.pug file in the views directory app.get("/html", (_, res) => { res.render("index", {title: "Hey", message: "Hello there!"}); }); ``` **Encore.ts** ```typescript import {api} from "encore.dev/api"; import Handlebars from "handlebars"; const html = `

Hello {{name}}!

`; // Making use of raw endpoints to serve dynamic templates. // https://encore.dev/docs/ts/primitives/raw-endpoints export const serveHTML = api.raw( {expose: true, path: "/html", method: "GET"}, async (req, resp) => { const template = Handlebars.compile(html); resp.setHeader("Content-Type", "text/html"); resp.end(template({name: "Simon"})); }, ); ``` ### Testing Express.js does not have built-in testing support. You can use libraries like [Vitest](https://vitest.dev/) and [Supertest](https://www.npmjs.com/package/supertest). With Encore.ts you are able to call the API endpoints directly in your tests, just like any other function. You then run the tests using the `encore test` command. Learn more in our [Testing docs](https://encore.dev/docs/ts/develop/testing). **Express.js** ```typescript import {describe, expect, test} from "vitest"; import request from "supertest"; import express from "express"; import getRequestExample from "../get-request-example"; /** * We need to add the supertest library to make fake HTTP requests to the Express.js app without having to * start the server. We also use the vitest library to write tests. */ describe("Express App", () => { const app = express(); app.use("/", getRequestExample); test("should respond with a greeting message", async () => { const response = await request(app).get("/hello/John"); expect(response.status).to.equal(200); expect(response.body).to.have.property("message"); expect(response.body.message).to.equal("Hello John!"); }); }); ``` **Encore.ts** ```typescript import {describe, expect, test} from "vitest"; import {dynamicPathParamExample} from "../get-request-example"; // This test suite demonstrates how to test an Encore route. // Run tests using the `encore test` command. describe("Encore app", () => { test("should respond with a greeting message", async () => { // You can call the Encore.ts endpoint directly in your tests, // just like any other function. const resp = await dynamicPathParamExample({name: "world"}); expect(resp.message).toBe("Hello world!"); }); }); ``` ### Database Express.js does not have built-in database support. You can use libraries like [pg-promise](https://www.npmjs.com/package/pg-promise) to connect to a PostgreSQL database but you also have to manage Docker Compose files for different environments. With Encore.ts, you create a database by importing `encore.dev/storage/sqldb` and calling `new SQLDatabase`, assigning the result to a top-level variable. Database schemas are defined by creating migration files. Each migration runs in order and expresses the change in the database schema from the previous migration. Encore.ts automatically provisions databases to match what your application requires. Encore.ts provisions databases in an appropriate way depending on the environment. When running locally, Encore creates a database cluster using Docker. In the cloud, it depends on the environment type: To query data, use the `.query` or `.queryRow` methods. To insert data, or to make database queryies that don't return any rows, use `.exec`. Learn more in our [Database docs](https://encore.dev/docs/ts/primitives/databases). **Express.js** ```typescript -- db.ts -- import express, {Request, Response} from "express"; import pgPromise from "pg-promise"; const app: Express = express(); // Connect to the DB with the credentials from docker-compose.yml const db = pgPromise()({ host: "localhost", port: 5432, database: "database", user: "user1", password: "user1@123", }); interface User { name: string; id: number; } // Get one User from DB app.get("/user/:id", async (req: Request, res: Response) => { const user = await db.oneOrNone( ` SELECT * FROM users WHERE id = $1 `, req.params.id, ); res.json({user}); }); -- docker-compose.yml -- version: '3.8' services: db: build: context: . dockerfile: Dockerfile.postgis # Use custom Dockerfile restart: always environment: POSTGRES_USER: user1 POSTGRES_PASSWORD: user1@123 POSTGRES_DB: database healthcheck: # This command checks if the database is ready, right on the source db server test: [ "CMD-SHELL", "pg_isready" ] interval: 5s timeout: 5s retries: 5 ports: - "5432:5432" volumes: - postgres_data_v:/var/lib/postgresql/data volumes: postgres_data_v: -- Dockerfile.postgis -- FROM postgres:latest # Install PostGIS extension RUN apt-get update \ && apt-get install -y postgis postgresql-12-postgis-3 \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* # To execute some initial queries, we can write queries in init.sql COPY init.sql /docker-entrypoint-initdb.d/ # Enable PostGIS extension RUN echo "CREATE EXTENSION IF NOT EXISTS postgis;" >> /docker-entrypoint-initdb.d/init.sqld ``` **Encore.ts** ```typescript -- db.ts -- import {api} from "encore.dev/api"; import {SQLDatabase} from "encore.dev/storage/sqldb"; // Define a database named 'users', using the database migrations in the "./migrations" folder. // Encore automatically provisions, migrates, and connects to the database. export const DB = new SQLDatabase("users", { migrations: "./migrations", }); interface User { name: string; id: number; } // Get one User from DB export const getUser = api( {expose: true, method: "GET", path: "/user/:id"}, async ({id}: { id: number }): Promise<{ user: User | null }> => { const user = await DB.queryRow` SELECT name FROM users WHERE id = ${id} `; return {user}; }, ); // Add User from DB export const addUser = api( { expose: true, method: "POST", path: "/user" }, async ({ name }: { name: string }): Promise => { await DB.exec` INSERT INTO users (name) VALUES (${name}) `; return; }, ); -- migrations/1_create_tables.up.sql -- CREATE TABLE users ( id SERIAL PRIMARY KEY, name TEXT NOT NULL UNIQUE ); ``` ### Logging Express.js does not have built-in support for logging. You can use libraries like [Winston](https://www.npmjs.com/package/winston) to log messages. Encore.ts offers built-in support for Structured Logging, which combines a free-form log message with structured and type-safe key-value pairs. Logging is integrated with the built-in [Distributed Tracing](https://encore.dev/docs/tracing) functionality, and all logs are automatically included in the active trace. **Encore.ts** ```typescript import log from "encore.dev/log"; log.error(err, "something went terribly wrong!"); log.info("log message", {is_subscriber: true}); ``` --- Source: https://encore.dev/docs/ts/how-to/nestjs Section: Guides (Encore.ts) # Use NestJS with Encore [Nest](https://docs.nestjs.com/) (NestJS) is a framework for building efficient, scalable TypeScript server-side applications. Nest aims to provide an application architecture out of the box which allows for effortless creation of highly testable, scalable, and loosely coupled and easily maintainable applications. Encore is not opinionated when it comes to application architecture, so you can use it together with NestJS to structure your business logic and Encore for creating backend primitives like APIs, Databases, and Cron Jobs. **Related example:** [encoredev/examples/ts/nestjs](https://github.com/encoredev/examples/tree/main/ts/nestjs) Encore.ts + NestJS example Create it locally with `encore app create --example=ts/nestjs`. ## Adding Encore to a NestJS project If you already have a NestJS project, you can add Encore to it by following these steps: 1. Run `encore app init` in the root of your project to create a new Encore application. 2. Add `encore.dev` as a dependency by running `npm install encore.dev`. 3. Add the following `paths` to your `tsconfig.json`: ```json -- tsconfig.json -- { "compilerOptions": { "paths": { "~encore/*": [ "./encore.gen/*" ] } } } ``` ## Standalone Nest application In order for Encore to be able to provision infrastructure resources, generate API documentation etc. we need to run our application using Encore. This means that we need to replace the ordinary Nest bootstrapping and instead run our Nest app as a [standalone application](https://docs.nestjs.com/standalone-applications). We do this by calling `NestFactory.createApplicationContext(AppModule)` and then selecting the modules/services we need: ```ts -- applicationContext.ts -- const applicationContext: Promise<{ catsService: CatsService }> = NestFactory.createApplicationContext(AppModule).then((app) => { return { catsService: app.select(CatsModule).get(CatsService, {strict: true}), // other services... }; }); export default applicationContext; ``` The `applicationContext` variable can then be used to access your Nest modules/services from your Encore your APIs. ## Defining an Encore service When running an app using Encore you need at least one [Encore service](https://encore.dev/docs/ts/primitives/services#defining-services). You can define a service in two ways: 1. Create a folder and inside that folder defining one or more APIs. Encore recognizes this as a service, and uses the folder name as the service name. 2. Add a file named `encore.service.ts` in a directory. The file must export a service instance, by calling `new Service`, imported from `encore.dev/service`: ```ts -- encore.service.ts -- import {Service} from "encore.dev/service"; export default new Service("my-service"); ``` Encore will consider this directory and all its subdirectories as part of the service. If you already have a Nest app then the easiest way to get going is to go with the second approach and add a `encore.service.ts` in the root of your app, then you do not need to change your existing folder structure. ## Replacing Nest controllers with Encore APIs If you already have a Nest app then you can keep most of your business logic (modules, services and providers) as is but in order for Encore to be able to manage your APIs, you need to replace your Nest controllers with Encore APIs. Let's assume you have a `cats/cats.controller.ts` in your Nest app that looks like this: ```ts -- cats/cats.controller.ts -- @Controller('cats') export class CatsController { constructor(private readonly catsService: CatsService) { } @Post() @Roles(['admin']) async create(@Body() createCatDto: CreateCatDto) { this.catsService.create(createCatDto); } @Get() async findAll(): Promise { return this.catsService.findAll(); } @Get(':id') findOne( @Param('id', new ParseIntPipe()) id: number, ) { return this.catsService.get(id); } } ``` When converting this to using Encore it would look like this: ```ts -- cats/cats.controller.ts -- export const findAll = api( {expose: true, method: 'GET', path: '/cats'}, async (): Promise<{ cats: Cat[] }> => { const {catsService} = await applicationContext; return {cats: await catsService.findAll()}; }, ); export const get = api( {expose: true, method: 'GET', path: '/cats/:id'}, async ({id}: { id: number }): Promise<{ cat: Cat }> => { const {catsService} = await applicationContext; return {cat: await catsService.get(id)}; }, ); export const create = api( {expose: true, auth: true, method: 'POST', path: '/cats'}, async (dto: CreateCatDto): Promise => { const {catsService} = await applicationContext; catsService.create(dto); }, ); ``` We use the `applicationContext` (that we defined above) to access our `catsService` and pass in the necessary parameters. Both Encore and Nest use the concept of a `service`. With Encore you define a service by creating a folder and inside that folder defining one or more APIs. Encore recognizes this as a service, and uses the folder name as the service name. When deploying, Encore will automatically provision the required infrastructure for each service. So in the example above we have a `cats` service with three APIs because `cats.controller.ts` is placed inside a folder named `cats`. ## Making use of other Encore features Encore also allows you to easily make use of other backend primitives in your Nest app, like [Databases](https://encore.dev/docs/ts/primitives/databases), [Cron Jobs](https://encore.dev/docs/ts/primitives/cron-jobs), [Pub/Sub & Queues](https://encore.dev/docs/ts/primitives/pubsub) and [Secrets](https://encore.dev/docs/ts/primitives/secrets). Take a look at our [Encore + NestJS example](https://github.com/encoredev/examples/tree/main/ts/nestjs) which uses both a PostgreSQL Database and an [Auth Handler](https://encore.dev/docs/ts/develop/auth) to authenticate incoming requests. ## Running your Encore app After those steps we are ready to run our app locally: ```shell $ encore run ``` You should see log messages about both Encore and Nest staring up. That means your local development environment is up and running and ready to take some requests! ### Open the Local Development Dashboard You can now start using your [Local Development Dashboard](https://encore.dev/docs/dev-dash). Open [http://localhost:9400](http://localhost:9400) in your browser to access it. The Local Development Dashboard is a powerful tool to help you move faster when you're developing new features. It comes with an API explorer, a Service Catalog with automatically generated documentation, and powerful observability features like [distributed tracing](https://encore.dev/docs/tracing). --- Source: https://encore.dev/docs/go/how-to/temporal Section: Guides (Encore.go) # Use Temporal with Encore [Temporal](https://temporal.io) is a workflow orchestration system for building highly reliable systems. Encore works great with Temporal, and this guide shows you how to integrate Temporal into your Encore application. ## Set up Temporal clusters You'll need at least two Temporal clusters: one for local development and one for cloud environments. We recommend using the [Temporal CLI](https://docs.temporal.io/cli) dev server for local development, and [Temporal Cloud](https://temporal.io/cloud) for cloud environments. Install the CLI with Homebrew (`brew install temporal`), or follow the [install instructions](https://docs.temporal.io/cli/setup-cli) for other platforms. ## Set up Temporal Workflow Next it's time to create a Temporal Workflow. We'll base this on the Temporal [Hello World](https://learn.temporal.io/getting_started/go/hello_world_in_go/) example. Create a new Encore service named `greeting`: ```go -- greeting/greeting.go -- package greeting import ( "context" "fmt" "go.temporal.io/sdk/client" "go.temporal.io/sdk/worker" "encore.dev" ) // Use an environment-specific task queue so we can use the same // Temporal Cluster for all cloud environments. var ( envName = encore.Meta().Environment.Name greetingTaskQueue = envName + "-greeting" ) //encore:service type Service struct { client client.Client worker worker.Worker } func initService() (*Service, error) { c, err := client.Dial(client.Options{}) if err != nil { return nil, fmt.Errorf("create temporal client: %v", err) } w := worker.New(c, greetingTaskQueue, worker.Options{}) err = w.Start() if err != nil { c.Close() return nil, fmt.Errorf("start temporal worker: %v", err) } return &Service{client: c, worker: w}, nil } func (s *Service) Shutdown(force context.Context) { s.client.Close() s.worker.Stop() } ``` Next it's time to define some workflows. These need to be in the same service, so add a new `workflow` package inside the `greeting` service, containing a workflow and activity definition in separate files: ```go -- greeting/workflow/workflow.go -- package workflow import ( "time" "go.temporal.io/sdk/workflow" ) func Greeting(ctx workflow.Context, name string) (string, error) { options := workflow.ActivityOptions{ StartToCloseTimeout: time.Second * 5, } ctx = workflow.WithActivityOptions(ctx, options) var result string err := workflow.ExecuteActivity(ctx, ComposeGreeting, name).Get(ctx, &result) return result, err } -- greeting/workflow/activity.go -- package workflow import ( "context" "fmt" ) func ComposeGreeting(ctx context.Context, name string) (string, error) { greeting := fmt.Sprintf("Hello %s!", name) return greeting, nil } ``` Then, go back to the `greeting` service and register the workflow and activity: ```go -- greeting/greeting.go -- // Import the package at the top: import "encore.app/greeting/workflow" // Add these lines to `initService`, below the call to `worker.New`: w.RegisterWorkflow(workflow.Greeting) w.RegisterActivity(workflow.ComposeGreeting) ``` Now let's create an Encore API that triggers this workflow. Add a new file `greeting/greet.go`: ```go -- greeting/greet.go -- package greeting import ( "context" "encore.app/greeting/workflow" "encore.dev/rlog" "go.temporal.io/sdk/client" ) type GreetResponse struct { Greeting string } //encore:api public path=/greet/:name func (s *Service) Greet(ctx context.Context, name string) (*GreetResponse, error) { options := client.StartWorkflowOptions{ ID: "greeting-workflow", TaskQueue: greetingTaskQueue, } we, err := s.client.ExecuteWorkflow(ctx, options, workflow.Greeting, name) if err != nil { return nil, err } rlog.Info("started workflow", "id", we.GetID(), "run_id", we.GetRunID()) // Get the results var greeting string err = we.Get(ctx, &greeting) if err != nil { return nil, err } return &GreetResponse{Greeting: greeting}, nil } ``` ## Run it locally Now we're ready to test it out. Start the Temporal dev server and your Encore application (in separate terminals): ```bash $ temporal server start-dev $ encore run ``` `temporal server start-dev` creates the `default` namespace, listens for gRPC on `localhost:7233`, and serves the Web UI at [http://localhost:8233](http://localhost:8233). Now try calling it, either from the [Local Development Dashboard](https://encore.dev/docs/dev-dash) or using cURL: ```bash $ curl 'http://localhost:4000/greeting/Temporal' {"Greeting": "Hello Temporal!"} ``` If you see this, it works! ## Run in the cloud To run it in the cloud, you will need to use Temporal Cloud or your own, self-hosted Temporal cluster. The easiest way to automatically pick up the correct cluster address is to use Encore's [config functionality](https://encore.dev/docs/go/develop/config). Add two new files: ``` -- greeting/config.go -- package greeting import "encore.dev/config" type Config struct { TemporalServer string } var cfg = config.Load[*Config]() -- greeting/config.cue -- package greeting TemporalServer: [ // These act as individual case statements if #Meta.Environment.Cloud == "local" { "localhost:7233" }, // TODO: configure this to match your own cluster address "my.cluster.address:7233", ][0] // Return the first value which matches the condition ``` Finally go back to `greeting/greeting.go` and update the `client.Dial` call to look like: ```go -- greeting/greeting.go -- client.Dial(client.Options{HostPort: cfg.TemporalServer}) ``` With that, Encore will automatically connect to the correct Temporal cluster, using a local cluster for local development and your cloud-hosted cluster for everything else. --- Source: https://encore.dev/docs/cli/cli-reference Section: Reference # CLI Reference > The Encore CLI lets you run your local environment and much more. ## Running #### Run Runs your application. ```shell $ encore run [--debug] [--watch=true] [--port=] [--listen=] [flags] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `-w, --watch` | Watch for changes and live-reload | `true` | | `--listen` | Address to listen on (e.g. `0.0.0.0:4000`) | | | `-p, --port` | Port to listen on | `4000` | | `--json` | Display logs in JSON format | `false` | | `-n, --namespace` | Namespace to use (defaults to active namespace) | | | `--color` | Whether to display colorized output | auto-detected | | `--redact` | Redact sensitive data in traces when running locally | `false` | | `-l, --level` | Minimum log level to display (`trace\|debug\|info\|warn\|error`) | | | `--debug` | Compile for debugging (`enabled\|break`) | | | `--browser` | Open local dev dashboard in browser on startup (`auto\|never\|always`) | `auto` | #### Test Tests your application. **Encore.ts:** Runs the test script defined in your `package.json`. ```shell $ encore test [flags] ``` **Encore.go:** Takes all the same flags as `go test`. ```shell $ encore test ./... [go test flags] ``` Additional flags recognized by `encore test`: | Flag | Description | | --- | --- | | `--codegen-debug` | Dump generated code (for debugging Encore's code generation) | | `--prepare` | Prepare for running tests without running them | | `--trace` | Write trace information about the parse and compilation process to a file | | `--no-color` | Disable colorized output | #### Check Compiles your application, starts the infrastructure it needs, and waits for it to start. Then it runs each command in the spec against the app, prints the results, and shuts the app down. With no spec, it just checks that the app starts. The exit code is 0 when every command succeeds, and 1 if the app fails to start or any command fails. ```shell $ encore check [spec] [flags] ``` The spec is one command per line, or several separated by `;`. Commands are `curl [args...]`, where the path starts with `/` and any further arguments are passed to curl. Pass the spec as an argument, or pipe it in: ```shell $ encore check $ encore check 'curl /ping' $ encore check 'curl /ping; curl /hello/world -i' $ encore check < spec.txt ``` A command fails if curl exits with an error or the response status is 400 or above. To check for an error status, such as a 401 without auth, add `--no-fail`: `encore check 'curl /admin --no-fail -i'`. **Flags** | Flag | Description | Default | | --- | --- | --- | | `--timeout` | How long to wait for the app to start before giving up | `1m0s` | #### Exec Runs executable scripts against the local Encore app. **Encore.ts:** Takes a command that it will execute with the local Encore app environment setup. ```shell $ encore exec -- ``` **Encore.go:** Compiles and runs a Go script with the local Encore app environment setup. ```shell $ encore exec [...args] ``` The command directory should contain Go files with `package main` and a `main` function. The additional arguments are passed directly to the built binary. **Flags** | Flag | Description | | --- | --- | | `-n, --namespace` | Namespace to use (defaults to active namespace) | **Example** Run a database seed script **Encore.ts:** ```shell $ encore exec -- npx tsx ./seed.ts ``` **Encore.go:** ```shell $ encore exec cmd/seed ``` ## App Commands to create and link Encore apps #### Clone Clone an existing Encore app from the Encore platform to your computer ```shell $ encore app clone [app-id] [directory] ``` Requires `encore auth login`. Clones into a directory named after the app ID by default, with a git remote named `encore`, so you can deploy changes with `git push encore`. #### Create Create a new Encore app ```shell $ encore app create [name] [flags] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `--example` | URL to example code to use | | | `-l, --lang` | Programming language to use for the app | | | `-r, --llm-rules` | Initialize the app with LLM rules for a specific tool | | | `--platform` | Whether to create the app with the Encore Platform | `true` | #### Init Register an existing local repo as a new app on the Encore platform ```shell $ encore app init [name] [flags] ``` Run `encore auth login` first: logged in, it writes a new `encore.app` and adds the `encore` git remote. Logged out, the app isn't registered on Encore Cloud. See [Already have a local app?](https://encore.dev/docs/platform/deploy/deploying#already-have-a-local-app) for the full steps. **Flags** | Flag | Description | | --- | --- | | `-l, --lang` | Programming language to use for the app | #### Link Link an existing local repo to an existing Encore app ```shell $ encore app link [app-id] [flags] ``` The app must already exist on Encore Cloud. `link` adds the `encore` git remote only if the repository has no remotes yet. Otherwise, add it with `git remote add encore encore://`. **Flags** | Flag | Description | | --- | --- | | `-f, --force` | Force link even if the app is already linked | ## Auth Commands to authenticate with Encore #### Login Log in to Encore ```shell $ encore auth login [flags] ``` **Flags** | Flag | Description | | --- | --- | | `-k, --auth-key` | Auth Key to use for login | #### Logout Logs out the currently logged in user ```shell $ encore auth logout ``` #### Signup Create a new Encore account ```shell $ encore auth signup ``` #### Whoami Show the current logged in user ```shell $ encore auth whoami ``` ## Daemon Encore CLI daemon commands #### Restart If you experience unexpected behavior, try restarting the daemon using: ```shell $ encore daemon ``` #### Env Outputs Encore environment information ```shell $ encore daemon env ``` ## Database Management Database management commands #### Connect to database via shell Connects to the database via psql shell Defaults to connecting to your local environment. Specify --env to connect to another environment. Use `--test` to connect to databases used for integration testing. Use `--shadow` to connect to the shadow database, used for database drift detection when using tools like Prisma. `--test` and `--shadow` imply `--env=local`. ```shell $ encore db shell [DATABASE_NAME] [--env=] [flags] ``` `encore db shell` defaults to read-only permissions. Use `--write`, `--admin` and `--superuser` flags to modify which permissions you connect with. **Flags** | Flag | Description | Default | | --- | --- | --- | | `-n, --namespace` | Namespace to use (defaults to active namespace) | | | `-e, --env` | Environment name to connect to | `local` | | `-t, --test` | Connect to the integration test database (implies --env=local) | `false` | | `--shadow` | Connect to the shadow database (implies --env=local) | `false` | | `--write` | Connect with write privileges | `false` | | `--admin` | Connect with admin privileges | `false` | | `--superuser` | Connect as a superuser | `false` | #### Connection URI Outputs a database connection string. Defaults to connecting to your local environment. Specify --env to connect to another environment. ```shell $ encore db conn-uri [] [--env=] [flags] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `-n, --namespace` | Namespace to use (defaults to active namespace) | | | `-e, --env` | Environment name to connect to | `local` | | `-t, --test` | Connect to the integration test database (implies --env=local) | `false` | | `--shadow` | Connect to the shadow database (implies --env=local) | `false` | | `--write` | Connect with write privileges | `false` | | `--admin` | Connect with admin privileges | `false` | | `--superuser` | Connect as a superuser | `false` | #### Proxy Sets up local proxy that forwards any incoming connection to the databases in the specified environment. ```shell $ encore db proxy [--env=] [flags] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `-n, --namespace` | Namespace to use (defaults to active namespace) | | | `-e, --env` | Environment name to connect to | `local` | | `-p, --port` | Port to listen on (defaults to a random port) | `0` | | `-t, --test` | Connect to the integration test database (implies --env=local) | `false` | | `--shadow` | Connect to the shadow database (implies --env=local) | `false` | | `--write` | Connect with write privileges | `false` | | `--admin` | Connect with admin privileges | `false` | | `--superuser` | Connect as a superuser | `false` | #### Reset Resets the databases for the given services. Use --all to reset all databases. ```shell $ encore db reset [flags] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `-n, --namespace` | Namespace to use (defaults to active namespace) | | | `--all` | Reset all services in the application | `false` | | `-t, --test` | Reset databases in the test cluster instead | `false` | | `--shadow` | Reset databases in the shadow cluster instead | `false` | ## Code Generation Code generation commands #### Generate client Generates an API client for your app. For more information about the generated clients, see [this page](https://encore.dev/docs/ts/cli/client-generation). By default, `encore gen client` generates the client based on the version of your application currently running in your local environment. You can change this using the `--env` flag and specifying the environment name. Use `--lang=` to specify the language. Supported language codes are: - `go`: A Go client using the net/http package - `typescript`: A TypeScript client using the in-browser Fetch API - `javascript`: A JavaScript client using the in-browser Fetch API - `openapi`: An OpenAPI spec ```shell $ encore gen client [] [--env=] [--lang=] [flags] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `-l, --lang` | Language to generate code for | | | `-o, --output` | Filename to write the generated client code to | | | `-e, --env` | Environment to fetch the API for | `local` | | `-s, --services` | Names of the services to include in the output | | | `-x, --excluded-services` | Names of the services to exclude in the output | | | `-t, --tags` | Names of endpoint tags to include in the output | | | `--excluded-tags` | Names of endpoint tags to exclude in the output | | | `--openapi-exclude-private-endpoints` | Exclude private endpoints from the OpenAPI spec | `false` | | `--ts:shared-types` | Import types from ~backend instead of re-generating them | `false` | | `--target` | An optional target for the client (`leap`) | | ## Logs Streams logs from your application ```shell $ encore logs [--env=prod] [--json] [flags] ``` **Flags** | Flag | Description | | --- | --- | | `-e, --env` | Environment name to stream logs from (defaults to the primary environment) | | `--json` | Whether to print logs in raw JSON format | | `-q, --quiet` | Whether to print initial message when the command is waiting for logs | ## Kubernetes Kubernetes management commands #### Configure Updates your kubectl config to point to the Kubernetes cluster(s) for the specified environment ```shell $ encore k8s configure --env=ENV_NAME ``` ## Secrets Management Secret management commands #### Set Set a secret value for a specific environment: ```shell $ encore secret set --env ``` Set a secret value for an environment type: ```shell $ encore secret set --type ``` Where `` defines which environment types the secret value applies to. Use a comma-separated list of `production`, `development`, `preview`, and `local`. Shorthands: `prod`, `dev`, `pr`. **Examples** Entering a secret directly in terminal: ```shell $ encore secret set --type dev MySecret Enter secret value: ... Successfully created secret value for MySecret. ``` Piping a secret from a file: ```shell $ encore secret set --type dev,local MySecret < my-secret.txt Successfully created secret value for MySecret. ``` Note that this strips trailing newlines from the secret value. #### List Lists secrets, optionally for a specific key ```shell $ encore secret list [keys...] ``` #### Delete Deletes a secret value ```shell $ encore secret delete ``` ## Namespaces Manage infrastructure namespaces for isolating local infrastructure. See [Infrastructure Namespaces](https://encore.dev/docs/infra-namespaces) for more details. #### List List infrastructure namespaces ```shell $ encore namespace list [--output=columns|json] ``` #### Create Create a new infrastructure namespace ```shell $ encore namespace create NAME ``` #### Delete Delete an infrastructure namespace ```shell $ encore namespace delete NAME ``` #### Switch Switch to a different infrastructure namespace. Subsequent commands will use the given namespace by default. Use `-` as the namespace name to switch back to the previously active namespace. ```shell $ encore namespace switch [--create] NAME ``` **Flags** | Flag | Description | | --- | --- | | `-c, --create` | Create the namespace before switching | ## Config Gets or sets configuration values for customizing the behavior of the Encore CLI. Configuration options can be set both for individual Encore applications, as well as globally for the local user. ```shell $ encore config [] [flags] ``` When running `encore config` within an Encore application, it automatically sets and gets configuration for that application. To set or get global configuration, use the `--global` flag. **Flags** | Flag | Description | | --- | --- | | `--all` | View all settings | | `--app` | Set the value for the current app | | `--global` | Set the value at the global level | ## Telemetry Reports the current telemetry status ```shell $ encore telemetry ``` #### Enable Enables telemetry reporting ```shell $ encore telemetry enable ``` #### Disable Disables telemetry reporting ```shell $ encore telemetry disable ``` ## MCP MCP (Model Context Protocol) commands for integrating with AI assistants. See [MCP](https://encore.dev/docs/mcp) for more details. #### Start Starts an SSE-based MCP session and prints the SSE URL ```shell $ encore mcp start [--app=] ``` #### Run Runs a stdio-based MCP session ```shell $ encore mcp run [--app=] ``` ## Random Utilities for generating cryptographically secure random data. #### UUID Generates a random UUID (defaults to version 4) ```shell $ encore rand uuid [-1|-4|-6|-7] ``` **Flags** | Flag | Description | | --- | --- | | `-1, --v1` | Generate a version 1 UUID | | `-4, --v4` | Generate a version 4 UUID (default) | | `-6, --v6` | Generate a version 6 UUID | | `-7, --v7` | Generate a version 7 UUID | #### Bytes Generates random bytes and outputs them in the specified format ```shell $ encore rand bytes BYTES [-f ] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `-f, --format` | Output format (`hex\|base32\|base32hex\|base32crockford\|base64\|base64url\|raw`) | `hex` | | `--no-padding` | Omit padding characters from base32/base64 output | `false` | #### Words Generates random 4-5 letter words for memorable passphrases ```shell $ encore rand words [--sep=SEPARATOR] NUM ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `-s, --sep` | Separator between words | ` ` (space) | ## Deploy Deploy an Encore app to a cloud environment. Most deploys happen with `git push encore`, or a push to GitHub if you've [linked a repository](https://encore.dev/docs/platform/integrations/github). Requires either `--commit` or `--branch` to be specified. ```shell $ encore alpha deploy --env= (--commit= | --branch=) [flags] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `--app` | App slug to deploy to (defaults to current app) | | | `-e, --env` | Environment to deploy to (required) | | | `--commit` | Commit SHA to deploy | | | `--branch` | Branch to deploy | | | `-f, --format` | Output format (`text\|json`) | `text` | ## Version Reports the current version of the encore application ```shell $ encore version ``` #### Update Checks for an update of encore and, if one is available, runs the appropriate command to update it. ```shell $ encore version update ``` ## Build Generates an image for your app, which can be used to [self-host](https://encore.dev/docs/self-host/build) your app. #### Docker Builds a portable Docker image of your Encore application. ```shell $ encore build docker IMAGE_TAG [flags] ``` **Flags** | Flag | Description | Default | | --- | --- | --- | | `--base` | Base image to build from | `scratch` | | `-p, --push` | Push image to remote repository | `false` | | `--cgo` | Enable cgo | `false` | | `--config` | Infra configuration file path | | | `--skip-config` | Do not read or generate an infra configuration file | `false` | | `--services` | Services to include in the image | | | `--gateways` | Gateways to include in the image | | | `--os` | Target operating system | `linux` | | `--arch` | Target architecture (`amd64\|arm64`) | `amd64` | ## LLM Rules Generate LLM rules in an existing app #### Init Initialize the LLM rules files ```shell $ encore llm-rules init [flags] ``` **Flags** | Flag | Description | | --- | --- | | `-r, --llm-rules` | Initialize the app with LLM rules for a specific tool (`cursor\|claudecode\|vscode\|agentsmd\|zed`) | --- Source: https://encore.dev/docs/cli/config-reference Section: Reference # Configuration Reference > Configuration options to customize the behavior of the Encore CLI. The Encore CLI has a number of configuration options to customize its behavior. Configuration options can be set both for individual Encore applications, as well as globally for the local user. Configuration options can be set using `encore config `, and options can similarly be read using `encore config `. When running `encore config` within an Encore application, it automatically sets and gets configuration for that application. To set or get global configuration, use the `--global` flag. ## Configuration files The configuration is stored in one ore more TOML files on the filesystem. The configuration is read from the following files, in order: ### Global configuration * `$XDG_CONFIG_HOME/encore/config` * `$HOME/.config/encore/config` * `$HOME/.encoreconfig` ### Application-specific configuration * `$APP_ROOT/.encore/config` Where `$APP_ROOT` is the directory containing the `encore.app` file. The files are read and merged, in the order defined above, with latter files taking precedence over earlier files. ## Configuration options #### run.browser Type: string
Default: auto
Must be one of: always, never, or auto Whether to open the Local Development Dashboard in the browser on `encore run`. If set to "auto", the browser will be opened if the dashboard is not already open. --- Source: https://encore.dev/docs/platform/integrations/api-reference Section: Reference # Encore Platform API Reference The Encore platform provides an API for programmatic access to control certain parts of the platform. We're working on expanding the set of features available over the API. Please reach out to us [on Discord](https://encore.dev/discord) if you have use cases where additional API functionality would be useful. The Base URL for the Encore platform API is `https://api.encore.dev`. ## Authentication All API calls require valid authentication, which is provided by sending an access token in the `Authorization` header, in the format `Authorization: Bearer ${ACCESS_TOKEN}`. You can retrieve an API access token from the OAuth Token endpoint, using an OAuth Client. An API access token expires after one hour. For continuous access, shortly before an API access token expires, request a new API access token from the Encore platform's OAuth token endpoint. OAuth client libraries in popular programming languages can handle the API access token generation and renewal. See the [OAuth Clients](https://encore.dev/docs/platform/integrations/oauth-clients) for more information on creating OAuth Clients. ## OAuth **Method**: `POST`
**Path**: `/api/oauth/token` #### Query Parameters | Parameter | Description | | ----------------- | -------------------------------------------------------------- | | **client_id** | The client id of the OAuth Client to generate a token for. | | **client_secret** | The client secret of the OAuth Client to generate a token for. | #### Response The API responds with a 2xx status code on successful creation of an API access token. ```typescript type Token = { // The access token itself. "access_token": string; // The access token expires after 1 hour (3600 seconds). "expires_in": 3600; // The actor the token belongs to, in this case the OAuth2 client id. actor: string; // Indicates the access token should be passed as a "Bearer" token in the Authorization header. "token_type": "Bearer"; } ``` ## Response Format Every endpoint except the OAuth token endpoint wraps its payload in an envelope: ```json { "ok": true, "data": { } } ``` Errors use the same envelope: ```json { "ok": false, "error": { // The error code, for example "validation" or "not_found". "code": "validation", // Extra information about the error, when there is any. // For validation errors it names the field at fault. "detail": { "field": "services", "type": "empty" }, // A human readable description, when there is one. "detailed_error": "" } } ``` Where this reference says an endpoint returns an object, that object is the value of `data`. ## Rollouts The Encore platform's deployment system consists of several phases: * A build phase * An infrastructure provisioning phase * A deployment phase These phases are combined into a unified entity called a *Rollout*. A rollout represents the coordinated process of rolling out a specific version of an Encore application. We use the term *rollout* to disambiguate from the *deployment phase*, which specifically refers to the last phase of the rollout process (where the version is being deployed onto the provisioned infrastructure). ### The Rollout Object The Rollout object represents the state of a rollout. ```typescript // The representation of a rollout. type Rollout = { // Unique id of the rollout. id: string; // The current status of the rollout. status: "pending" | "queued" | "running" | "completed"; // What the conclusion was of the rollout (when status is "completed"). // If the status is not "completed" the conclusion is "pending". conclusion: "pending" | "canceled" | "failure" | "success"; // When the rollout was queued, started, and completed. queued_at: Date | null; started_at: Date | null; completed_at: Date | null; // Information about the various rollout phases. // A phase is null until the rollout reaches it, and the infra phase // is null when the rollout has no infrastructure change. // See type definitions below. build: RolloutPhase | null; infra: RolloutPhase | null; deploy: RolloutPhase | null; } // Common structure of the various rollout phases. type RolloutPhase = { // Unique id of the phase. id: string; // The current status of the rollout phase. status: Status; // What the conclusion was of the phase. conclusion: Conclusion; // When the phase was queued, started, and completed. queued_at: Date | null; started_at: Date | null; completed_at: Date | null; } // The current status and conclusion of a build. // If the status is not "completed" the conclusion is "unknown". type BuildStatus = "queued" | "running" | "completed"; type BuildConclusion = "unknown" | "canceled" | "failure" | "success"; // The current status and conclusion of an infra change. // The "proposed" status means the change is awaiting human approval. // The "rejected" conclusion means a human rejected the proposed infra change. type InfraStatus = "pending" | "proposed" | "queued" | "running" | "completed"; type InfraConclusion = "unknown" | "canceled" | "failure" | "rejected" | "success"; // The current status and conclusion of a deploy. // If the status is not "completed" the conclusion is "unknown". type DeployStatus = "pending" | "queued" | "running" | "completed"; type DeployConclusion = "unknown" | "canceled" | "failure" | "success"; ``` ### Triggering a rollout **Method**: `POST`
**Path**: `/api/apps/${APP_ID}/envs/${ENV_NAME}/rollouts` #### Path Parameters | Parameter | Description | | ------------ | -------------------------------------------------------------------------------------- | | **APP_ID** | The id of the Encore application to trigger a rollout for. | | **ENV_NAME** | The name of the environment to trigger a rollout for. Use `@primary` for the app's primary environment. | #### JSON Request Body ```json { // Which commit to roll out. Provide exactly one of the three. // The order of precedence is latest_deploy, then sha, then branch. // // The commit hash to roll out. "sha": "abc123...", // The name of the branch to roll out the latest commit from. "branch": "main", // Roll out the commit currently deployed to the environment, again. "latest_deploy": false, // Optional. Set to skip running tests during build. "skip_tests": false, // Optional. Set to force a rebuild instead of reusing a cached build. "force_rebuild": false, // Optional. Deploy only these services, leaving the rest of the // environment on the revision it is already running. "services": ["users", "billing"], // Optional. Leave the gateways on the revision they are running. // Only valid together with "services". "skip_gateways": false, // Optional. Deploy what the rollout covers even if the platform // considers it already up to date. "force_redeploy": false } ``` Exactly one of `sha`, `branch` and `latest_deploy` must be provided; if more than one is given, `latest_deploy` takes precedence over `sha`, which takes precedence over `branch`. All other fields are optional: the booleans default to `false`, and an omitted `services` deploys the whole environment. **Naming the commit.** `latest_deploy` resolves the commit of the environment's current deploy and starts a new rollout from it, on the same path as a commit or branch rollout. It is the deployed commit, not the most recent rollout's, which may have failed or may still be in flight. If nothing is deployed to the environment the request fails with a 400. **Partial deploys.** `services` restricts the deploy phase only: infrastructure is still provisioned for the whole environment and database migrations still run. Services that are not named keep running the revision they are already on. Two restrictions apply to the set of services: * Services that share a process must be deployed together, since a container runs one image. Naming some but not all of the services in a group fails the rollout during planning, with an error naming the ones left out. Set `process_per_service` on the environment to be able to deploy them apart. * Service names are not checked when the rollout is created — the build is what says which services exist — so an unknown name also fails the rollout during planning rather than being rejected here. `skip_gateways` only means something alongside `services`, and sending it without a non-empty `services` is rejected with a 400. Gateways are deployed by default, because a gateway left behind does not know about the endpoints the new revision adds. Holding them back instead avoids applying the new revision's view of routes to services that are still serving the old one. `force_redeploy` overrides the platform's record of what is already current, and nothing else. In particular it does not widen the scope: a rollout restricted with `services` still redeploys only those. **Deduplication.** If an active rollout already exists for the same environment and commit, it is returned instead of a new one being created. Rollouts count as the same request only if they also agree on `services`, `skip_gateways` and `force_redeploy`, so a narrower redeploy of the same commit is not folded into an earlier one. #### Response The API responds with a 2xx status code on successful creation of a new rollout. On success it returns a **Rollout** object as its JSON response payload, representing the current state of the newly created rollout. ### Retrieving a rollout **Method**: `GET`
**Path**: `/api/apps/${APP_ID}/rollouts/${ROLLOUT_ID}` #### Path Parameters | Parameter | Description | | -------------- | ------------------------------------------------------------ | | **APP_ID** | The id of the Encore application the rollout belongs to. | | **ROLLOUT_ID** | The id of the rollout to retrieve, in the form `"roll_..."`. | #### Response The API responds with a 2xx status code on successful retrieval of the rollout. On success it returns a **Rollout** object as its JSON response payload, representing the current state of the requested rollout. ### Approving infrastructure changes Approves the proposed infrastructure changes for a rollout, allowing infrastructure provisioning to proceed. **Method**: `POST`
**Path**: `/api/apps/${APP_ID}/rollouts/${ROLLOUT_ID}/approve-infra` #### Path Parameters | Parameter | Description | | -------------- | ------------------------------------------------------------- | | **APP_ID** | The id of the Encore application the rollout belongs to. | | **ROLLOUT_ID** | The id of the rollout to approve, in the form `"roll_..."`. | #### Response The API responds with a 2xx status code when the request is successful. On success it returns a **Rollout** object as its JSON response payload, representing the rollout after the approval attempt. When the infrastructure change is awaiting approval, its status changes from `proposed` to `queued`. ### Rejecting infrastructure changes Rejects the proposed infrastructure changes for a rollout and cancels the rollout. **Method**: `POST`
**Path**: `/api/apps/${APP_ID}/rollouts/${ROLLOUT_ID}/reject-infra` #### Path Parameters | Parameter | Description | | -------------- | ------------------------------------------------------------- | | **APP_ID** | The id of the Encore application the rollout belongs to. | | **ROLLOUT_ID** | The id of the rollout to reject, in the form `"roll_..."`. | #### Response The API responds with a 2xx status code when the request is successful. On success it returns a **Rollout** object as its JSON response payload, representing the rollout after the rejection attempt. When the infrastructure change is awaiting approval, its status changes to `completed` with the `rejected` conclusion. Rejecting the infrastructure changes causes the rollout to be canceled. ### Describing infrastructure changes Describes the infrastructure changes a rollout proposes: the same plan the dashboard shows when asking for approval. The id to pass is the `id` of the rollout's `infra` phase. **Method**: `GET`
**Path**: `/api/apps/${APP_ID}/infra-changes/${INFRA_CHANGE_ID}` #### Path Parameters | Parameter | Description | | -------------------- | ------------------------------------------------------------------------ | | **APP_ID** | The id of the Encore application the infrastructure change belongs to. | | **INFRA_CHANGE_ID** | The id of the infrastructure change, in the form `"infchg_..."`. | #### Response The API responds with a 2xx status code on successful retrieval, and with 404 if the change does not exist or belongs to another application. ```typescript type InfraChange = { // Unique id of the infrastructure change. id: string; // The status and conclusion of the change, as on a rollout's infra phase. status: InfraStatus; conclusion: InfraConclusion; // When the change was queued, started, and completed. queued_at: Date | null; started_at: Date | null; completed_at: Date | null; // The environment the change applies to, in the form "env_...". env_id: string; // The rollout the change belongs to, in the form "roll_...". // Null for a change that was not made as part of a rollout. rollout_id: string | null; // What kind of change this is. type: "provision" | "deploy" | "cleanup" | "delete" | "restore" | "custom" | "fetch"; // The commit the change was planned from. commit_hash: string; // Whether any of the changes below restarts a running resource. requires_restart: boolean; // The individual resource changes making up the plan. changes: ResourceChange[]; } type ResourceChange = { // What is happening to the resource. action: "create" | "modify" | "delete" | "deploy" | "other"; // Whether applying this change restarts the resource. requires_restart: boolean; // Whether the change has to be applied at the cloud provider. provider_update: boolean; // The resource fields that are changing, for example "MemoryMiB". changed_fields: string[]; // The resource being changed. See below. resource: object; } ``` Only changes that have to be applied at the cloud provider are listed; bookkeeping-only changes are left out, the same way the approval screen leaves them out. The `resource` object uses the dashboard's resource schema, so its field names are camelCase rather than snake_case, and a configuration value that is changing is reported as a `{"current": ..., "requested": ...}` pair — the requested value, not a snapshot of a change that has already been applied. Costs are not included. ## Member Management The Encore platform provides APIs for managing application members, including inviting users, listing members, and updating member roles. ### Member Object ```typescript type Member = { // The user's email address email: string; // The member's role in the application role: "owner" | "reader" | "writer" | "none"; // When the member was invited to the application invited: timestamp; // When the member accepted the invitation accepted: timestamp; // When the membership expires expires: timestamp; // The member's username username: string; // The member's full name full_name: string; // The member's picture URL picture_url: string; } ``` ### Available Roles - **owner**: Full control over the application - **writer**: Can write application resources - **reader**: Can read application resources - **none**: Used to revoke access to an application ### Invite Member Invite a new member to an Encore application. **Method**: `POST`
**Path**: `/api/apps/${APP_ID}/member` #### Path Parameters | Parameter | Description | | ---------- | ---------------------------------------------- | | **APP_ID** | The id of the Encore application. | #### JSON Request Body ```typescript { // The email address of the user to invite "email": string; // The role to assign to the invited member "role": "owner" | "reader" | "writer" | "none"; } ``` #### Response The API responds with a 2xx status code on successful invitation. On success it returns a **Member** object as its JSON response payload, representing the newly invited member. ### List Members Retrieve a list of all members for an Encore application. **Method**: `GET`
**Path**: `/api/apps/${APP_ID}/members` #### Path Parameters | Parameter | Description | | ---------- | ---------------------------------------------- | | **APP_ID** | The id of the Encore application. | #### Response The API responds with a 2xx status code on success. On success it returns an array of **Member** objects as its JSON response payload, representing all current members and pending invites for the application. ```typescript type Response = Member[]; ``` ### Update Member Role Update the role of an existing member. **Method**: `PUT`
**Path**: `/api/apps/${APP_ID}/members` #### Path Parameters | Parameter | Description | | ---------- | ---------------------------------------------- | | **APP_ID** | The id of the Encore application. | #### JSON Request Body ```typescript { // The email address of the member to update "email": string; // The new role to assign to the member "role": "owner" | "reader" | "writer" | "none"; } ``` #### Response The API responds with a 2xx status code on successful update. #### Error Cases - **403 Forbidden**: Insufficient permissions to manage members - **409 Conflict**: Attempting to remove the last owner (error detail: "last_owner") - **404 Not Found**: Member not found --- Source: https://encore.dev/docs/community/get-involved Section: Community # Community > Join the most pioneering developer community! Developers building with Encore are forward-thinkers, who are working on exciting and innovative applications. We rely on this group's feedback, and contributions to the Open Source project, to improve Encore for developers everywhere. Getting involved is a fantastic way of finding support and inspiration among peers. Everyone is welcome in the Encore community, and we hope you to get involved too! ## Get involved There are many ways to get involved. Here's where you can start straight away.

Contribute on GitHub

Use GitHub to report bugs, feedback on proposals, or contribute your ideas.

Join Discord

Connect with fellow Encore developers, ask questions, or just hang out!

Follow on Twitter

Follow Encore on Twitter to keep up with the latest. Share what you've built to help spread the word about the project. ### Contribute to the project Want to make a contribution to Encore? Great, start by reading about the different [ways to contribute](https://encore.dev/docs/community/contribute). ### Feedback on the Roadmap [The Encore Roadmap](https://encore.dev/roadmap) is public. It's open to your comments, feature requests, and you can vote on existing entries. ## Community Governance We recommend everyone read the [Community Principles](https://encore.dev/docs/community/principles). If you need assistance, have concerns, or have questions for the Community team, please email us at [support@encore.dev](mailto:support@encore.dev). --- Source: https://encore.dev/docs/community/contribute Section: Community # Ways to contribute > Guidelines for contributing to Encore We’re so excited that you are interested in contributing to Encore! All contributions are welcome, and there are several valuable ways to contribute. ### Open Source Project If you want to contribute to the Encore Open Source project, you can submit a pull request on [GitHub](https://github.com/encoredev/encore/pulls). ### Report issues If you have run into an issue or think you’ve found a bug, please report it via the [issue tracker](https://github.com/encoredev/encore/issues). ### Add or update docs If there’s something you think would be helpful to add to the docs or if there’s something that seems out of date, we appreciate your input. You can view the docs and contribute fixes or improvements directly in [GitHub](https://github.com/encoredev/encore/tree/main/docs). You can also email your feedback to us at [hello@encore.dev](mailto:hello@encore.dev). ### Blog posts If you’ve built something cool using Encore, we’d really like you to talk about it! We love it when developers share their projects on blogs and on Twitter. Use the hashtag **#builtwithencore** and we’ll have an easier time finding your work. – We might also showcase it on the [Encore Twitter account](https://twitter.com/encoredotdev)! ### Meetups & Workshops Organizing a meetup or workshop is a great way to connect with other developers using Encore. It can also be a great first step in trying out Encore for development in your company or other professional organization. If you want help with organizing or planning an event, please don’t hesitate to reach out to us via email at [hello@encore.dev](mailto:hello@encore.dev). --- Source: https://encore.dev/docs/community/open-source Section: Community # Open Source > Encore is Open Source Software We believe Open Source is key to a long-term sustainable and prosperous technology community. Encore builds on Open Source software, and is largely Open Source itself. ## License Encore's Backend Framework, parser, and compiler are Open Source under Mozilla Public License 2.0. > The MPL is a simple copyleft license. The MPL's "file-level" copyleft is designed to encourage contributors to share modifications they make to your code, while still allowing them to combine your code with code under other licenses (open or proprietary) with minimal restrictions. You can learn more about MPL 2.0 on [the official website](https://www.mozilla.org/en-US/MPL/2.0/FAQ/). ## Contribute Contributions to improve Encore are very welcome. Contribute to Encore on [GitHub](https://github.com/encoredev/encore). --- Source: https://encore.dev/docs/community/principles Section: Community # Community principles > Everyone belongs in the Encore community Everyone is welcome in the Encore community, and it is of utmost importance to us that everyone is able to feel at home and contribute. Therefore we as maintainers, and you as a contributor, must pledge to make participation in our community a harassment-free experience for everyone, regardless of: age, body size, disability, ethnicity, gender identity, level of experience, nationality, personal appearance, race, religion, or sexual identity. ### Code of Conduct To this end, the Encore community is guided by the [Contributor Covenant 2.0 Code of Conduct](https://www.contributor-covenant.org/version/2/0/code_of_conduct/) to ensure everyone is welcome and able to participate. --- Source: https://encore.dev/docs/community/submit-template Section: Community # Submit a Template > Your contributions help other developers build [Templates](https://encore.dev/templates) help and inspire developers to build applications using Encore. You are welcome to contribute your own templates! Two types of templates that are especially useful: - **Starters:** Runnable Encore applications for others to use as is, or take inspiration from. - **Bits:** Re-usable code samples to solve common development patterns or integrate Encore applications with third-party APIs and services. ## Submit your contribution Contribute a template by submitting a Pull Request to the [Open Source Examples Repo](https://github.com/encoredev/examples): `https://github.com/encoredev/examples` ### Submitting Starters Follow these steps to submit a **Starter**: 1. Fork the repo. 2. Create a new folder in the root directory of the repo, this is where you will place your template. — Use a short folder name as your template will be installable via the CLI, like so: `encore app create APP-NAME --example=` 3. Include a `README.md` with instructions for how to use the template. We recommend following [this format](https://github.com/encoredev/examples/blob/8c7e33243f6bfb1b2654839e996e9a924dcd309e/uptime/README.md). Once your Pull Request has been approved, it may be featured on the [Templates page](https://encore.dev/templates) on the Encore website. ### Submitting Bits Follow these steps to submit your **Bits**: 1. Fork the repo. 2. Create a new folder inside the `bits` folder in the repo and place your template inside it. Use a short folder name as your template will soon be installable via the CLI. 3. Include a `README.md` with instructions for how to use the template. Once your Pull Request has been approved, it may be featured on the [Templates page](https://encore.dev/templates) on the Encore website. ## Contribute from your own repo If you don't want to contribute code to the examples repo, but still want to be featured on the [Templates page](https://encore.dev/templates), please contact us at [hello@encore.dev](mailto:hello@encore.dev). ## Dynamic Encore AppID In most cases, you should avoid hardcoding an `AppID` in your template's source code. Instead, use the notation `{{ENCORE_APP_ID}}`. When a developer creates an app using the template, `{{ENCORE_APP_ID}}` will be dymically replaced with their new and unique `AppID`, meaning they will not need to make any manual code adjustments.