talktalk

Talk 0.1.106 gets cooperative tasks, parallel workers, channels, macros that write checked Talk, a native C route, real file and text APIs, cached packages, a new website, and a compiler living inside Obsidian.

This is the issue where Talk stops looking like a language feature list and starts behaving like a place to work. Waiting code keeps its direct style. Macros can extend the source language without escaping the checker. Programs can move from the VM to a self-contained native executable, while the same compiler now powers examples on talktalk.fyi and code fences in an Obsidian vault. Around that center, conversions, files, Unicode text, package caching, time, arithmetic, and documentation all got markedly more practical.

Talk now supports cooperative tasks without adding async functions, await expressions, or future-valued return types. Waiting is an effect. An ordinary function calls sleep, channel receive, or another suspending operation in direct style and still returns its ordinary result. The inferred effect row records that the function may wait, so the type system sees the suspension even though every caller does not have to change color.

Every executable starts inside an implicit cooperative root task. 'spawn adds a child, and 'yield voluntarily lets another ready task run. A task also yields when it waits for a deadline or channel. If the root finishes first, the scheduler keeps going until its children finish. On one worker these tasks do not run simultaneously; a switch happens at a suspension operation, which keeps ordinary local state comprehensible.

use task::{ sleep }

func reminder() -> String {
	sleep(.milliseconds(100))
	"time is up"
}

'spawn { print(reminder()) }

print("root continues")
'yield()

The same waiting operations also have blocking outer handlers. Without a cooperative handler, a worker can block and resume normally; when the compiler can prove the reachable program does not need scheduling, it may choose that cheaper route. coop::run remains available for an explicit nested scheduling scope, but the top level no longer needs a ceremonial run wrapper.

CPU work gets a separate, structured route. parallel_run starts one worker for each input, waits for every worker, and returns results in input order. Jobs and results must conform to Send. Native targets move them through shared-memory workers, while the reference VM gives workers isolated machines and structurally copies transferable values. Worker handles never escape, nested joins can help run queued work, and targets without threads may preserve the same source behavior sequentially.

Channels connect both kinds of work. They are multi-producer and single-consumer; cloning a Sender creates another producer, and recv returns .none once every sender is gone and the queue is empty. Bounded channels add backpressure with an atomic capacity reservation. select_recv waits on either of two receivers, deterministically choosing the left when both are ready without removing the losing value. Timers use a monotonic clock, so wall-clock changes cannot make a sleep return early.

Declarative macros are now hygienic balanced-token templates rather than expression-only substitution. A braced body can contain binders, control flow, free names, and types; the invocation position decides whether the expansion must parse as an expression, declaration, nominal member, pattern, or type. Names written in the template resolve at the definition site, spliced syntax keeps the caller's context, and generated code proceeds through normal type, effect, ownership, and exhaustiveness checking.

A useful macro can add scope-exit cleanup without making every caller name and preserve a guard:

struct ScopeExit {
	let action: () -> ()
}

extend ScopeExit: Deinit {
	consuming func deinit() { self.action() }
}

macro defer($cleanup) {
	let _scope_exit = ScopeExit {
		$cleanup
	}
}

@defer { print("doin this second") }
print("doin this first")

@defer expands to a hygienic hidden binding. Its Deinit hook runs when read_file leaves scope, including through its tail return, so the file closes after the read. A helper function could construct ScopeExit, but every caller would have to bind and retain the returned guard; discarding it would run cleanup immediately. The macro supplies that otherwise easy-to-forget lifetime pattern, and multiple uses do not collide on _scope_exit.

Declaration wrappers add the attached form #[name] or #[name(args)]. A wrapper receives the following declaration as typed syntax and may replace it, remove it, or return a structured failure. It can inspect and rebuild a declaration's name and body while preserving token provenance and hygiene. Adjacent wrappers compose from the declaration outward, and a replacement is checked in the actual file, block, struct, enum, protocol, or extension context where the original appeared.

Procedural expression macros live in deterministic *.macro.tlk services. They receive one balanced token tree and return category-checked syntax, run under fixed budgets, and cannot reach inline IR or #unsafe. This is compile-time extension without granting arbitrary host filesystem, environment, process, or network access. The bundled @html macro is substantial enough to deserve its own section.

The html package now ships a procedural @html macro written in Talk. It parses a compact template language at compile time and expands it into an ordinary checked Talk expression returning Markup. Element names are written directly, #content supplies an ID, .page appends a class, braces add children and a closing tag, and a semicolon emits only an opening tag for void elements.

use html::{ html }

let name = "<Ada & friends>"
let links = ["docs", "playground"]

let page = @html {
	main #content .page {
		h1 { "Hello, " (name) }
		ul {
			@for link in links {
				li { a href={ "/" (link) } { (link) } }
			}
		}
	}
}

print(page.into_string())

Parenthesized Talk expressions interpolate values, and braced templates can construct dynamic attribute values. Attributes may be always present, conditional Booleans, or optional values omitted for .none. Template control flow includes @if, @else if, @if let, destructuring @for, scoped @let, and exhaustive @match; the generated bindings, patterns, effects, and ownership all go back through the normal checker.

Rendering is escaped by default. Values pass through RenderHtml, whose default implementation escapes ampersands, angle brackets, both kinds of quote, and other text through its Showable representation. Nested Markup and explicitly constructed PreEscaped fragments bypass that pass, making the trust boundary visible rather than treating every string as safe HTML. Markup.into_string() transfers the finished output when it is time to write or return it.

The boundary is intentionally HTML syntax, not browser policy. The macro accepts custom element names and does not validate element nesting, duplicate attributes, URL schemes, ARIA rules, JavaScript, or CSS. Application code still owns those decisions. What the macro guarantees is narrower and useful: the template parses during compilation, generated Talk is type-checked, and ordinary interpolated values cannot silently become tags.

The new home is talktalk.fyi. The front page is still willing to make jokes at Alan Kay, but its examples now run against the WebAssembly compiler in the page. Editable blocks expose Run, Lower, and Format controls, surface inferred information and diagnostics, and can accumulate earlier named blocks so a tour can build one program without duplicating every declaration.

The site now carries a full guide from getting started through effects, ownership, collections, macros, concurrency, packages, testing, the toolchain, and interop. Type inference, MIR, and bytecode each get a reference appendix. Generated API pages cover Core and the bundled packages, with the manual and reference sharing one navigation system. The site build checks runnable manual examples instead of trusting prose that may have drifted away from the compiler.

The browser is not pretending to be a separate toy language. The WASM surface calls the same formatting, highlighting, analysis, lowering, and execution machinery used elsewhere, and the site ships the compiler's Core artifact with it. Procedural package macros still belong to the CLI rather than the browser playground, but ordinary language examples now have a much shorter route from documentation to experiment.

The TalkTalk Obsidian plugin puts that same WASM compiler behind tlk fences. In Reading mode it uses parser-backed highlighting, prints diagnostics beneath a block, and adds a Run button. Execution happens in a dedicated Web Worker with timeout protection, so a runaway Talk loop does not take Obsidian's interface with it.

Source mode and Live Preview add diagnostics, inferred-type hover, and scope and member completion through a persistent analysis worker. A command formats the fence under the cursor. Accumulating and norun metadata mirror the website: execution, diagnostics, hover, and completion can see the accumulated source while formatting and highlighting stay focused on the current block.

Release 0.5.8 is available from GitHub with main.js, manifest.json, and styles.css assets suitable for BRAT or a manual vault install. The plugin even has an explicit WASM reload path, which is a healthy admission that placing a compiler inside a note-taking application may occasionally require turning the compiler off and on again.

obsidian plugin

A declared Into<Target> conformance can now satisfy an ordinary value crossing implicitly. When a value enters an annotated binding, a free-function argument, or a return slot with a different known type, the checker can insert the consuming .into() that the programmer would otherwise write. Generic conformance rows participate after specialization, so this is not limited to one-off concrete wrappers.

struct Word {
	let text: String
}

extend Word: Into<String> {
	consuming func into() -> String { self.text }
}

func shout(_ message: String) -> String {
	message + "!"
}

let greeting: String = Word(
	text: "hello"
)
print(shout(Word(text: "hi")))

The conversion is selected by the expected target type and a unique declared row. Reflexive T: Into<T> witnesses exist for generic code but do not invent conversions between unrelated types, and a missing declared row remains a normal type mismatch. Because into consumes its receiver, an implicit conversion transfers ownership through the witness rather than duplicating a payload behind the type checker's back.

The C backend is no longer a partial experiment. talk c emits one self-contained C translation unit from the optimized, register-allocated program used by the bytecode route. talk build --native feeds that unit to the host C compiler and produces an executable. The result contains no bytecode and links no Talk runtime library; the generated file carries its runtime and depends only on libc.

$ talk build --native app.tlk -o app
$ ./app
$ talk c app.tlk > app.c

--cc selects a compiler, --cflag passes flags through, and --keep-c preserves the generated unit. Cross-target builds go through zig cc when --target is supplied. From x86-64 Linux, the project has verified x86-64 and AArch64 musl, AArch64 macOS, RISC-V 64 musl, x86-64 Windows GNU, and 32-bit x86 musl targets; the musl outputs are static.

Every MIR instruction is covered. The backend document records 229 compiling corpus programs that agree with the interpreter, with none rejected for a backend limitation. Closures, indirect calls, effect handlers, managed memory, globals, host I/O, heap regions, cells, and existentials all make the trip. Effects unwind through an explicit return-status protocol and shadow frame identity rather than setjmp, while functions that never manipulate continuations avoid that bookkeeping.

The performance result is large but should be read correctly. Whole benchmark programs measured from 6x to 210x faster than the VM, while the 10-million-iteration scalar loop reached about 450x on execution alone. The backend notes that this mostly identifies roughly 100 cycles of VM dispatch cost per bytecode instruction, not miraculous C generation. It is useful native output and a map of where the VM can improve at the same time.

The compiler can also emit C library artifacts with exported wrappers, a generated header and manifest, namespaced lifecycle calls, and status returns that keep a Talk trap from terminating the host process. The self-hosted frontend is already a consumer: its fixed-point MIR becomes C and is compiled into production compiler builds. In 0.1.106, MIR, bytecode, VM, C, native runtime, FFI, and WASM components also moved into explicit workspace crates, clarifying the backend boundaries without changing source behavior.

The fs package now provides Path, Directory, DirectoryEntry, and an open File handle. Paths can append components, normalize . and .. lexically, expand ~, become absolute against the current working directory, or ask the host for canonical symlink-resolved spelling. Directory traversal distinguishes files, directories, and symbolic links and does not follow symlinks during recursive walks.

File.open accepts .r, .w, and .rw. Reads can consume the remainder of a file or request one bounded chunk; writes loop across partial host writes; seek supports offsets from the start, current position, or end; tell and size expose position and metadata. Operations return Result with specific .open, .read, .write, .seek, and .stat errors carrying the host's negative errno.

String now has both an encoding surface and a deep Unicode surface. from_bytes and decoding_utf8 validate input, with UTF8Error reporting the first malformed byte offset; decoding_utf8_lossy replaces maximal malformed subsequences with U+FFFD. encoded_utf8 returns an independent byte array, while utf8() remains the borrowed byte view. Substring and static strings share common StringMethods without pretending that bytes, Unicode scalars, grapheme clusters, and owned storage are the same thing.

The everyday API now includes first and last character access, provenance-checked indexes and ranges, prefixes and suffixes, searching, occurrence counts, splitting and partitioning, line and Unicode word segmentation, trimming, joining, replacement and removal, repetition, reversal, mapping, filtering, padding, and tab expansion. Case conversion includes lowercase, uppercase, titlecase, capitalization, and case folding. Normalization covers NFC, NFD, NFKC, and NFKD, with queries for each form.

StringBuilder handles repeated construction without producing a chain of intermediate strings. TextCursor adds marks, peeking, advancing, restoring, slicing, predicate-based consumption, and literal consumption for parsers. Indexes carry source fingerprints so an index from unrelated text cannot quietly address a different string. Talk's original promise that iteration means extended grapheme clusters now has enough surrounding API to be useful rather than merely correct at parties.

Clean package libraries now compile into shared cached module images. The cache includes the exported module facts, typed bodies, and source closure, keyed by manifest and source contents, transitive library keys, the session module identity, and the compiler content stamp. Locked dependencies use it, and a root package's own library is consumed through the same module boundary by talk run, talk test, and the REPL.

That changes the edit-test loop materially. In the measured parser package, a test-only edit dropped a release talk test run from about 520 milliseconds to 210 milliseconds because roughly 1,000 lines of unchanged library source no longer had to be parsed, resolved, and checked again. Editing the library invalidates its image; editing only a test does not. Only clean results are stored, atomic rename makes concurrent writes safe, and a bounded number of compiler-stamped versions can coexist.

Artifacts live under $XDG_CACHE_HOME/talk or ~/.cache/talk, so they survive checkout changes and work for distributed compiler binaries. talk check deliberately stays fused because workspace checking must diagnose the actual source. If the root library is broken, run and test also fall back to the old fused compile so diagnostics do not disappear behind a failed cache build.

Ordinary signed Int addition, subtraction, multiplication, negation, and division now trap on overflow; division also traps on zero. Code that wants validation can use checked_add, checked_sub, checked_mul, checked_div, and checked_neg, which return an optional. Code that truly wants modulo-2^64 behavior can say unchecked_* explicitly. The VM and native route share the operation-specific traps, and optimizers preserve an overflowing operation even when its result is unused.

Core also gains Duration and Instant. Durations have constructors from nanoseconds through weeks, arithmetic, comparisons, ratios, and trimmed h:mm:ss display. Instant.now() reads a process-relative monotonic clock; only differences and additions are meaningful, not calendar dates. This clock drives the new task timers across the VM, WASM, native runtime, and deterministic test host.

Core modules and bundled package modules have moved to lowercase file and import identities, and the standard library is now organized as actual packages including fs, os, task, coop, dict, net, http, html, syntax, and testing. Generated docs follow those package boundaries. Untitled editor buffers now receive LSP analysis, hover presentation was rebuilt, the website's WASM is compressed, and the bootstrap artifacts now include a cached Core image so browser and editor integrations do not have to rediscover the language every time they wake up.

Twenty-three commits and releases from 0.1.87 through 0.1.106 fit into the twelve days behind this issue. The through-line is not simply volume. Talk's effects, ownership, modules, and self-hosted compiler are now being used to make concrete user surfaces: tasks instead of a concurrency sketch, syntax services instead of a macro proposal, files and text instead of raw host calls, native programs instead of backend diagrams, and documentation that can run the code it is explaining.