Lakeday docs

Instances

What you choose when you create an instance, how you reach it, and how you stop, start and resize it.

An instance is one Walleye server that Lakeday runs for you. It has its own database name, its own object storage and its own tokens. Nothing is shared with any other instance.

Shape and cache tier

You choose two things.

Shape is how many nodes serve the instance.

  • ramp is one node. Each write is durable in object storage before it is acknowledged.
  • launch is three nodes behind the same connection. Each write commits at a quorum of two of the three before it is acknowledged. Each table is owned by one node, so losing a node loses no acknowledged write, and that node’s tables stop answering until it is back.

Cache tier is how much memory and disk each node has for hot data. Tables can be larger than the cache; colder data is read from object storage when it is asked for.

Cacheramp nodelaunch node
small1 shared vCPU, 2 GiB, 8 GB disk2 dedicated vCPUs, 8 GiB, 32 GB disk
medium2 shared vCPUs, 4 GiB, 16 GB disk4 dedicated vCPUs, 16 GiB, 64 GB disk
large4 shared vCPUs, 8 GiB, 32 GB disk8 dedicated vCPUs, 32 GiB, 128 GB disk
xlarge8 shared vCPUs, 16 GiB, 64 GB disk16 dedicated vCPUs, 64 GiB, 256 GB disk

Start on ramp small. The rates are on Billing.

How the cache fills and how many tables fit is covered in the Walleye documentation under cache tiers.

Connecting

Every instance is reached at one endpoint, https://db.lakeday.ai, and named as the LanceDB database: db://<instance id>. Both are shown in the Connect section of the instance page. Pass the database as the URI and the endpoint as host_override (hostOverride in JavaScript):

db = lancedb.connect("db://<instance id>", api_key="<token>",
                     host_override="https://db.lakeday.ai")

The connection stays the same when you stop, start or resize the instance.

When a request goes unanswered

The endpoint sends a query, and an insert, again on its own when the instance does not begin to answer within a few seconds: both give the same result however often they arrive, because rows are written by their primary key. If three tries go unanswered you get one of two answers, each with an x-walleye-route-error header:

  • 503 upstream-unavailable for a query: nothing changed, retry it.
  • 504 outcome-unknown for an insert: the rows may or may not have been written. Sending the same rows again is safe; they are stored once.

Creating, dropping or indexing a table, and ingesting, are sent once. A 502 outcome-unknown from the instance itself means the same as the 504: the request reached it or may have, and whether it was applied is not known.

Access tokens

A client proves who it is with an access token made for that instance. Make them on the instance page with New token. Each token has:

  • a name, so you can tell them apart;
  • one or more scopes: data:read to query, data:write to add and change rows, data:manage to create and drop tables;
  • an optional expiry.

Making or revoking a token takes every scope it carries: your role must hold the matching data permission (see Roles).

The value is shown once. Only a hash of it is kept, so a lost token cannot be shown again: revoke it and make another.

Give each writer and reader its own token with only the scopes it needs. Revoking a token takes effect within 30 seconds and restarts nothing.

Stop, start and resize

Instances are always on until you stop them.

  • Stop finishes in-flight commits and checkpoints every table, then takes the instance down. A stopped instance keeps all its data and bills storage only.
  • Start brings it back on the same connection with every acknowledged write.
  • Resize changes the shape, the cache tier or both. It is accepted only while the instance is stopped: stop it, resize it, start it. Moving from ramp to launch keeps every table.

Delete

Deleting an instance destroys it and every table in its object storage. It cannot be undone. Its unbilled usage is settled first; see Billing.