Shale

A compact project bundler for CC:Tweaked. Shale turns a complete directory project into one executable Lua file, including its Lua modules and assets. The source tree is not needed at runtime: modules are loaded from memory and their normal return values are preserved, so the result can be either a program or a library.

Shale is aimed at projects which are too large to distribute comfortably on CC:Tweaked's default computers. Minification, scope-aware local renaming, tree-shaking, LZSS compression, and Huffman coding can be combined, while the generated file remains ordinary executable Lua.

Quick start

Run the standalone file directly:

wget run <raw-github-url>/shale.lua <source> [target] [options]

For example:

wget run <raw-github-url>/shale.lua Basalt dist/basalt.lua --entry src/main.lua

source is the directory to collect, target is the single Lua file to create, and --entry is relative to source. A release build will usually enable all optional optimization stages:

wget run <raw-github-url>/shale.lua MyProject release.lua --entry src/main.lua --rename --compress --tree-shake

The resulting program can be copied without its original project directory:

shell.run("release.lua")

If the entry returns a library API, loading the generated file returns that same value:

local api = dofile("release.lua")

Available options:

--entry <path>     Project-relative Lua entry file
--exclude <glob>   Exclude a path or glob (repeatable)
--rename           Shorten local identifiers using lexical scopes
--no-rename        Disable identifier renaming from Shalefile.lua
--compress         Compress the complete generated bundle
--no-compress      Disable compression from Shalefile.lua
--chain <n>        Compression effort, 1..512 (default 128)
--tree-shake       Drop modules unreachable from the entry
--no-tree-shake    Disable tree shaking from Shalefile.lua
--keep <glob>      Keep a module or Lua path glob (repeatable)
--max-size <size>  Fail if the bundle exceeds this size
--banner <text>    Prepend text to the bundle as a comment
--banner-file <p>  Prepend the contents of a file as a comment
--minify           Enable minification from Shalefile.lua
--no-minify        Preserve Lua source instead of minifying it
-h, --help         Show help
--version          Show the Shale version

If the target is inside the source directory, Shale automatically excludes that target from collection. The generated file is syntax-checked before the target is written.

Choosing the entry

The entry is the Lua file Shale executes after installing the in-memory module loader. Root-level main.lua, init.lua, and startup.lua are detected automatically, in that order. If a project has multiple possible starting points, always pass --entry.

Choose the code which actually starts the program or returns the public library API. An installer or a bootstrap which searches the filesystem and calls loadfile on the rest of the source tree is usually not the right single-file entry; choose the module it eventually launches instead. Modules using local require = ... are supported and receive Shale's bundled require.

Project paths become dotted module names:

core/input.lua    -> require("core.input")
core/ui/init.lua  -> require("core.ui")

Shalefile.lua

Projects can place a declarative Shalefile.lua in their root:

return {
    entry = "src/main.lua",
    output = "dist/application.lua",

    minify = true,
    rename = true,
    compress = true,
    treeShake = true,
    chain = 128,

    exclude = {
        "tests/**",
        "docs/**",
        "**/*.spec.lua",
    },

    keep = {
        "plugins/**",
        "modules/optional.lua",
    },

    maxSize = "900KiB",

    bannerFile = "LICENSE",
}

The project can then be built without repeating its settings:

shale MyProject

output is resolved relative to the project directory. An explicit target path takes precedence over it:

shale MyProject release.lua

CLI switches override scalar settings such as compress, rename, and maxSize. Command-line --exclude and --keep rules are appended to the lists from the Shalefile. Negative switches make configured optimisation stages easy to disable for debugging:

shale MyProject debug.lua --no-compress --no-rename

Shalefiles run with an empty environment and must return a plain configuration table. This keeps builds deterministic and prevents configuration loading from performing filesystem or network operations.

Recommended workflow

Start with an inspectable build when integrating an existing project:

shale MyProject debug.lua --entry src/main.lua --no-minify --no-rename --no-compress --no-tree-shake

Once that bundle runs correctly, enable the release stages:

shale MyProject release.lua --entry src/main.lua --rename --compress --tree-shake

If a Shalefile enables optimizations, the negative switches temporarily disable them without editing project configuration. Compression only affects startup and storage; disabling it is therefore the easiest way to inspect a generated bundle or traceback during integration.

Lexer

local lexer = require("shale.lexer")
local tokens = lexer.lex(source, {
    sourceName = "src/main.lua",
})

for _, token in ipairs(tokens) do
    print(token.kind, token.text, token.line, token.column)
end

Significant tokens are returned by default. Comments and whitespace can be retained when a caller needs exact source reconstruction:

local tokens = lexer.lex(source, {
    trivia = true,
})

Each token contains exact source text, inclusive byte offsets, a one-based start position, and an exclusive end position.

Safe minifier

local minifier = require("shale.minifier")
local output, stats = minifier.minify(source, {
    sourceName = "src/main.lua",
})

print(("%d -> %d bytes"):format(
    stats.inputBytes,
    stats.outputBytes
))

The minifier removes comments and unnecessary whitespace without renaming variables or rewriting literals. It re-tokenizes its output and rejects the result if any significant token differs from the original source. Quoted strings and long-bracket strings are kept as tokens rather than minified internally, so Lua source, templates, or other data stored inside a string are not modified.

Scope-aware local renaming

local renamer = require("shale.renamer")
local output, stats = renamer.rename(source, {
    sourceName = "src/main.lua",
})

The renamer parses lexical scopes before shortening local declarations, parameters, loop variables, and all corresponding references. Globals, member names, table keys, labels, string contents, implicit method self, and _ENV semantics are preserved. Project builds enable this additional stage with --rename; it cannot be combined with --no-minify.

Project collection

local project = require("shale.project")
local bundle = project.collect("MyProject", {
    entry = "src/main.lua",
    exclude = {
        "tests/**",
        "dist/**",
    },
})

The collector recursively discovers a source directory, sorts files deterministically, minifies each Lua module independently, preserves assets byte-for-byte, resolves an entry module, and assigns stable module IDs. main.lua, init.lua, and startup.lua are recognized automatically, while an explicit entry always takes precedence.

Path rules support * and ? within one path component and ** across any number of directories. Rules without wildcards continue to match the named file or directory and everything below that directory.

Uncompressed single-file bundles

local bundler = require("shale.bundler")
local source, stats = bundler.build(bundle)

The generated Lua file contains all processed modules, a cached in-memory require, the selected entry point, and embedded assets. Bundled modules can read assets explicitly:

local bundle = require("shale.bundle")

if bundle.exists("assets/config.json") then
    local contents = bundle.read("assets/config.json")
end

for _, path in ipairs(bundle.files()) do
    print(path)
end

bundle.entry() returns the project-relative entry path. Assets are preserved byte-for-byte, but Shale does not replace CC:Tweaked's global fs API: fs.open("assets/config.json") still refers to a real file on the computer. Read bundled, read-only assets through shale.bundle; save files, databases, downloaded content, and other writable data should remain external.

Modules which use the Basalt-style local require = ... convention are detected and receive Shale's bundled require as their first argument. Standard modules receive the requested module name and path through .... Missing bundled modules fall back to the host require.

Compression

--compress applies bounded-window LZSS compression to the complete generated bundle and embeds the binary stream as printable Base85. The bundle is syntax-checked before it is packed, so a compressed build cannot hide a broken bundle behind a loadable bootstrap. At startup the bootstrap verifies the Adler-32 checksum of the packed payload and the restored byte length before loading anything, and yields periodically so large bundles do not exceed CC:Tweaked's limit on uninterrupted execution.

For a smaller output, combine local renaming and compression:

shale MyProject build.lua --entry src/main.lua --rename --compress

The matcher selects matches lazily and searches up to --chain candidates per position, defaulting to 128. Raising it trades build time for a smaller file and changes nothing about the format:

shale MyProject build.lua --rename --compress --chain 512

Tokens are then entropy coded. Literals and match lengths share one alphabet, so no per-token flag bits are needed, and both that alphabet and the distance high byte get canonical Huffman codes. Shale builds the coded and the flat stream, charges the coded one for the larger decoder it needs, and ships whichever bundle comes out smaller — so small projects are never penalised for carrying a decoder they cannot repay. The CLI reports which one it chose.

The compressed payload is restored in memory every time the program starts. This reduces disk and transfer size, not the amount of Lua source which must eventually be compiled. Larger bundles therefore trade some startup time and temporary memory for a much smaller file on disk.

Tree shaking

--tree-shake keeps only the modules reachable from the entry, following require calls with literal names:

shale MyProject build.lua --entry src/main.lua --tree-shake

Assets are always kept, because nothing in the source says which of them a module will ask for. A module whose require target is computed at runtime cannot be traced; Shale reports every such call so you can add the modules it cannot see:

shale MyProject build.lua --tree-shake --keep widgets.button

Keep rules accept exact module names as before. Rules containing wildcards match project-relative Lua paths, which is useful for plugin directories:

shale MyProject build.lua --tree-shake --keep "plugins/**"

Size budgets

maxSize and --max-size place a hard limit on the final executable bundle:

shale MyProject build.lua --max-size 900KiB

The limit is checked after bundling and compression but before the target is written. Shale reports the remaining headroom for successful builds. Plain numbers are bytes; KB and MB use decimal units, while KiB and MiB use binary units.

Tested projects

These measurements are snapshots from real CraftOS-PC console runs and are not fixed compression guarantees:

Project	Collected Lua	Final file	Runtime check
Basalt	398.6 KiB	61,275 bytes	API, frame, label, theme, animation
Obsidian 1.0.0	459.8 KiB	68,501 bytes	Full nine-page showcase

Both builds used minification, local renaming, tree-shaking, and compression. The final size depends on source structure, assets, keep rules, and compression chain depth.

Building the standalone file

The modular source under src/ is itself bundled into the distributable shale.lua:

lua tools/build.lua