Jobbig

[ˈjɔbːɪɡ] the swedish word for bothersome

Notes about development

Jobbig is currently in early development, but should be usable above version 0.2. Below I will outline which features I am using in production, which features are tested and which features exist but are currently untested.

Versioning will look like this:

• During 0.1 I solidified the main API.
• I bumped 0.2 up to 0.3 when I rewrote the event plugin.
• During 0.3 I will roll out this to a few projects I am working on.
• During 0.4 I will write unit tests for core concepts.
• During 0.5 I will write type performance tests.
• During 0.6 I will write proper integration tests

During 0.x you should install the exact package version, as there can be breaking changes. I will try to keep this to a minimum.

Properly tested

I will be writing tests during version 0.2, but have currently not started this.

Features me or others verified to be working reliably

• the schedule and publish function without scheduledAt
• the use, on and handle functions
• the sqs plugin
• the drizzle store

Not properly tested at all

• the schedule and publish function with scheduledAt
• steps
• retries

Quick Start

npm install @jobbig/core       # npm
yarn add @jobbig/core          # yarn
bun add @jobbig/core           # bun
pnpm add @jobbig/core          # pnpm

There is also an optional drizzle store, this package might be moved into the core library at some point.

npm install @jobbig/drizzle       # npm
yarn add @jobbig/drizzle          # yarn
bun add @jobbig/drizzle           # bun
pnpm add @jobbig/drizzle          # pnpm

import { Job, Jobbig } from "@jobbig/core";
import { ServerPlugin } from "@jobbig/core/plugins";
import { LocalStore } from "@jobbig/local";
import { z } from "zod";

// 1) Define a job with steps
const processData = Job({
  id: "process-data",
  schema: z.object({
    input: z.number(),
    output: z.number().optional(),
  }),
  run: async ({ ctx }) => {
    await ctx.step.run("preprocess", async () => {
      console.log("Preprocessing input...");
    });

    await ctx.step.run("process", async () => {
      console.log("Processing data...");
    });

    await ctx.step.run("save", async () => {
      console.log("Saving results...");
    });
  },
});

// 2) Instantiate a store
const store = LocalStore({});

// 3) Create Jobbig, register plugins/jobs, and start the server worker
const jobbig = Jobbig({
  store,
  jobs: [processData],
}).use(ServerPlugin());

// 4) Schedule runs
await jobbig.schedule({
  jobId: "process-data",
  data: { input: 42 },
});

// 5) Start processing
jobbig.server();

Concepts

See Concepts for a full overview of key interfaces: Worker, Orchestrator, Runner, Job, Step, Run, and Store.

Development

# Install dependencies
bun install

# Build all packages
bun run build

# Run the example
cd examples/local
bun run index.ts

Roadmap

After making sure the core library is stable and works well with some good base plugins and tests, the main focus will be to make a studio like drizzle studio, where one can browse, reschedule jobs/events, view statistics about events and other niceties.

License

MIT