After some unexpected weekends of downtime looking after sick toddlers, Iβm happy to re-introduce top-bun
v7.
Re-introduce? Well, you may remember @siteup/cli
, a spiritual offshoot of sitedown
, the static site generator that turned a directory of markdown into a website.
Whats new with top-bun
v7?
Letβs dive into the new feature, changes and additions in top-bun
7.
Rename to top-bun
@siteup/cli
is now top-bun
.
As noted above, @siteup/cli
was a name hack because I didnβt snag the bare npm
name when it was available, and someone else had the genius idea of taking the same name. Hey it happens.
I described the project to Chat-GPT and it recommended the following gems:
quick-brick
web-erect
OK Chat-GPT, pretty good, I laughed, but Iβm not naming this web-erect
.
The kids have a recent obsession with Wallace & Gromit and we watched a lot of episodes while she was sick. Also Iβve really been enjoying π₯ bread themes so I decided to name it after Wallace & Gromitβs bakery βTop Bunβ in their hit movie βA Matter of Loaf and Deathβ.
A Docs Website
top-bun
now builds itβs own repo into a docs website. Itβs slightly better than the GitHub README.md view, so go check it out! It even has a real domain name so you know its for real.
- π top-bun.org
css
bundling is now handled by esbuild
esbuild
is an amazing tool. postcss
is a useful tool, but its slow and hard to keep up with. In top-bun
, css
bundling is now handled by esbuild
.
css
bundling is now faster and less fragile, and still supports many of the same transforms that siteup
had before. CSS nesting is now supported in every modern browser so we donβt even need a transform for that. Some basic transforms and prefixes are auto-applied by setting a relatively modern browser target.
esbuild
doesnβt support import chunking on css yet though, so each css
entrypoint becomes its own bundle. If esbuild
ever gets this optimization, so will top-bun
. In the meantime, global.css
, style.css
and now layout.style.css
give you ample room to generally optimize your scoped css loading by hand. Itβs simpler and has less moving parts!
Multi-layout support
You can now have more than one layout!
In prior releases, you could only have a single root
layout that you customized on a per-page basis with variables.
Now you can have as many layouts as you need.
They can even nest.
Check out this example of a nested layout from this website. Itβs named article.layout.js
and imports the root.layout.js
. It wraps the children
and then passes the results to root.layout.js
.
// article.layout.js
import { html } from 'uhtml-isomorphic'
import { sep } from 'node:path'
import { breadcrumb } from '../components/breadcrumb/index.js'
import defaultRootLayout from './root.layout.js'
export default function articleLayout (args) {
const { children, ...rest } = args
const vars = args.vars
const pathSegments = args.page.path.split(sep)
const wrappedChildren = html`
${breadcrumb({ pathSegments })}
<article class="article-layout h-entry" itemscope itemtype="http://schema.org/NewsArticle">
<header class="article-header">
<h1 class="p-name article-title" itemprop="headline">${vars.title}</h1>
<div class="metadata">
<address class="author-info" itemprop="author" itemscope itemtype="http://schema.org/Person">
${vars.authorImgUrl
? html`<img height="40" width="40" src="${vars.authorImgUrl}" alt="${vars.authorImgAlt}" class="u-photo" itemprop="image">`
: null
}
${vars.authorName && vars.authorUrl
? html`
<a href="${vars.authorUrl}" class="p-author h-card" itemprop="url">
<span itemprop="name">${vars.authorName}</span>
</a>`
: null
}
</address>
${vars.publishDate
? html`
<time class="published-date dt-published" itemprop="datePublished" datetime="${vars.publishDate}">
<a href="#" class="u-url">
${(new Date(vars.publishDate)).toLocaleString()}
</a>
</time>`
: null
}
${vars.updatedDate
? html`<time class="dt-updated" itemprop="dateModified" datetime="${vars.updatedDate}">Updated ${(new Date(vars.updatedDate)).toLocaleString()}</time>`
: null
}
</div>
</header>
<section class="e-content" itemprop="articleBody">
${typeof children === 'string'
? html([children])
: children /* Support both uhtml and string children. Optional. */
}
</section>
<!--
<footer>
<p>Footer notes or related info here...</p>
</footer>
-->
</article>
${breadcrumb({ pathSegments })}
`
return defaultRootLayout({ children: wrappedChildren, ...rest })
}
Layout styles and js bundles
With multi-layout support, it made sense to introduce two more style and js bundle types:
- Layout styles: styles that load on every page that uses a layout
- Layout bundles: js bundles that load on every page that uses a layout
Prior the global.css
and global.client.js
bundles served this need.
Layouts and Global Assets live anywhere
Layouts, and global.client.js
, etc used to have to live at the root of the project src
directory. This made it simple to find them when building, and eliminated duplicate singleton errors, but the root of websites is already crowded. It was easy enough to find these things anywhere, so now you can organize these special files in any way you like. Iβve been using:
layouts
: A folder full of layoutsglobals
: A folder full of the globally scoped files
Template files
Given the top-bun
variable cascade system, and not all website files are html, it made sense to include a templating system for generating any kind of file from the global.vars.js
variable set. This lets you generate random website βsidefilesβ from your site variables.
It works great for generating RSS feeds for websites built with top-bun
. Here is the template file that generates the RSS feed for this website:
import pMap from 'p-map'
import jsonfeedToAtom from 'jsonfeed-to-atom'
/**
* @template T
* @typedef {import('@siteup/cli').TemplateAsyncIterator<T>} TemplateAsyncIterator
*/
/** @type {TemplateAsyncIterator<{
* siteName: string,
* siteDescription: string,
* siteUrl: string,
* authorName: string,
* authorUrl: string,
* authorImgUrl: string
* layout: string,
* publishDate: string
* title: string
* }>} */
export default async function * feedsTemplate ({
vars: {
siteName,
siteDescription,
siteUrl,
authorName,
authorUrl,
authorImgUrl
},
pages
}) {
const blogPosts = pages
.filter(page => ['article', 'book-review'].includes(page.vars.layout) && page.vars.published !== false)
.sort((a, b) => new Date(b.vars.publishDate) - new Date(a.vars.publishDate))
.slice(0, 10)
const jsonFeed = {
version: 'https://jsonfeed.org/version/1',
title: siteName,
home_page_url: siteUrl,
feed_url: `${siteUrl}/feed.json`,
description: siteDescription,
author: {
name: authorName,
url: authorUrl,
avatar: authorImgUrl
},
items: await pMap(blogPosts, async (page) => {
return {
date_published: page.vars.publishDate,
title: page.vars.title,
url: `${siteUrl}/${page.pageInfo.path}/`,
id: `${siteUrl}/${page.pageInfo.path}/#${page.vars.publishDate}`,
content_html: await page.renderInnerPage({ pages })
}
}, { concurrency: 4 })
}
yield {
content: JSON.stringify(jsonFeed, null, ' '),
outputName: 'feed.json'
}
const atom = jsonfeedToAtom(jsonFeed)
yield {
content: atom,
outputName: 'feed.xml'
}
yield {
content: atom,
outputName: 'atom.xml'
}
}
Page Introspection
Pages, Layouts and Templates can now introspect every other page in the top-bun
build.
You can now easily implement any of the following:
- RSS feeds
- Auto-populating index feeds
- Tag systems
Pages, Layouts and Templates receive a pages
array that includes PageData instances for every page in the build. Variables are already pre-resolved, so you can easily filter, sort and target various pages in the build.
Full Type Support
top-bun
now has full type support. Itβs achieved with types-in-js
and it took a ton of time and effort.
The results are nice, but Iβm not sure the juice was worth the squeeze. top-bun
was working really well before types. Adding types required solidifying a lot of trivial details to make the type-checker happy. I donβt even think a single runtime bug was solved. It did help clarify some of the more complex types that had developed over the first 2 years of development though.
The biggest improvement provided here is that the following types are now exported from top-bun
:
LayoutFunction<T>
PostVarsFunction<T>
PageFunction<T>
TemplateFunction<T>
TemplateAsyncIterator<T>
You can use these to get some helpful auto-complete in LSP supported editors.
types-in-js
not Typescript
This was the first major dive I did into a project with types-in-js
support.
My overall conclusions are:
types-in-js
provides a superior development experience to developing in.ts
by eliminating the development loop build step.- JSDoc and
types-in-js
are in conflict with each other.types-in-js
should win, its better than JSDoc in almost every way (but you still use both). - Most of the JSDoc auto-generating documentation ecosystem doesnβt support
types-in-js
. Find something that consumes the generated types instead of consuming the JSDoc blocs. - There are a number of rough edges around importing types.
- The final api documentation features are nice.
- The internal types feel good, but were mostly a waste of time being introduced after the fact.
- I wish there was a way to strictly introduce types on just the public interfaces and work back, but this is challenging because many details come from deep within the program.
- Typescript puts upper limits on the dynamism normally allowed with JS. Itβs a superset syntax, that forces a subset of language functionality.
@ts-ignore
liberally. Take a pass or two to remove some later.
Handlebars support in md
and html
Previously, only js
pages had access to the variable cascade inside of the page itself. Now html
and md
pages can access these variables with handlebars placeholders.
## My markdown page
Hey this is a markdown page for {{ vars.siteName }} that uses handlebars templates.
Locally shipped default styles
Previously, if you opted for the default layout, it would import mine.css from unpkg. This worked, but went against the design goal of making top-bun
sites as reliable as possible (shipping all final assets to the dest folder).
Now when you build with the default layout, the default stylesheet (and theme picker js code) is built out into your dest
folder.
Built-in browser-sync
@siteup/cli
previously didnβt ship with a development server, meaning you had to run one in parallel when developing. This step is now eliminated now that top-bun
ships browser-sync
. browser-sync
is one of the best Node.js development servers out there and offers a bunch of really helpful dev tools built right in, including scroll position sync so testing across devices is actually enjoyable.
If you arenβt familiar with browser-sync, here are some screenshots of fun feature:
top-bun
eject
top-bun
now includes an --eject
flag, that will write out the default layout, style, client and dependencies into your src
folder and update package.json
. This lets you easily get started with customizing default layouts and styles when you decide you need more control.
β default-layout % npx top-bun --eject
top-bun eject actions:
- Write src/layouts/root.layout.mjs
- Write src/globals/global.css
- Write src/globals/global.client.mjs
- Add mine.css@^9.0.1 to package.json
- Add uhtml-isomorphic@^2.0.0 to package.json
- Add highlight.js@^11.9.0 to package.json
Continue? (Y/n) y
Done ejecting files!
The default layout is always supported, and its of course safe to rely on that.
Improved log output πͺ΅
The logging has been improved quite a bit. Here is an example log output from building this blog:
> top-bun --watch
Initial JS, CSS and Page Build Complete
bret.io/src => bret.io/public
βββ¬ projects
β βββ¬ websockets
β β βββ README.md: projects/websockets/index.html
β βββ¬ tron-legacy-2021
β β βββ README.md: projects/tron-legacy-2021/index.html
β βββ¬ package-automation
β β βββ README.md: projects/package-automation/index.html
β βββ page.js: projects/index.html
βββ¬ jobs
β βββ¬ netlify
β β βββ README.md: jobs/netlify/index.html
β βββ¬ littlstar
β β βββ README.md: jobs/littlstar/index.html
β βββ page.js: jobs/index.html
β βββ zhealth.md: jobs/zhealth.html
β βββ psu.md: jobs/psu.html
β βββ landrover.md: jobs/landrover.html
βββ¬ cv
β βββ README.md: cv/index.html
β βββ style.css: cv/style-IDZIRKYR.css
βββ¬ blog
β βββ¬ 2023
β β βββ¬ reintroducing-top-bun
β β β βββ README.md: blog/2023/reintroducing-top-bun/index.html
β β β βββ style.css: blog/2023/reintroducing-top-bun/style-E2RTO5OB.css
β β βββ¬ hello-world-again
β β β βββ README.md: blog/2023/hello-world-again/index.html
β β βββ page.js: blog/2023/index.html
β βββ page.js: blog/index.html
β βββ style.css: blog/style-NDOJ4YGB.css
βββ¬ layouts
β βββ root.layout.js: root
β βββ blog-index.layout.js: blog-index
β βββ blog-index.layout.css: layouts/blog-index.layout-PSZNH2YW.css
β βββ blog-auto-index.layout.js: blog-auto-index
β βββ blog-auto-index.layout.css: layouts/blog-auto-index.layout-2BVSCYSS.css
β βββ article.layout.js: article
β βββ article.layout.css: layouts/article.layout-MI62V7ZK.css
βββ globalStyle: globals/global-OO6KZ4MS.css
βββ globalClient: globals/global.client-HTTIO47Y.js
βββ globalVars: global.vars.js
βββ README.md: index.html
βββ style.css: style-E5WP7SNI.css
βββ booklist.md: booklist.html
βββ about.md: about.html
βββ manifest.json.template.js: manifest.json
βββ feeds.template.js: feed.json
βββ feeds.template.js-1: feed.xml
βββ feeds.template.js-2: atom.xml
[Browsersync] Access URLs:
--------------------------------------
Local: http://localhost:3000
External: http://192.168.0.187:3000
--------------------------------------
UI: http://localhost:3001
UI External: http://localhost:3001
--------------------------------------
[Browsersync] Serving files from: /Users/bret/Developer/bret.io/public
Copy watcher ready
Support for mjs
and cjs
file extensions
You can now name your page, template, vars, and layout files with the mjs
or cjs
file extensions. Sometimes this is a necessary evil. In general, set your type
in your package.json
correctly and stick with .js
.
Whatβs next for top-bun
The current plan is to keep sitting on this feature set for a while. But I have some ideas:
- Tell the world about it?
- Server routes with fastify (or expose a plugin to slot into an existing server)?
- Deeper integration with
uhtml
? - More web-component examples?
top-bun
is already one of the best environments for implementing sites that use web-components. Page bundles are a perfect place to register components! - Comparisons with other tools in the βenterprise-jsβ ecosystem?
If you try out top-bun
, I would love to hear about your experience. Do you like it? Do you hate it? Open an discussion item. or reach out privately.
History of top-bun
OK, now time for the story behind top-bun
aka @siteup/cli
.
I ran some experiments with orthogonal tool composition a few years ago. I realized I could build sophisticated module based websites by composing various tools together by simply running them in parallel.
What does this idea look like? See this snippet of a package.json
:
{ "scripts": {
"build": "npm run clean && run-p build:*",
"build:css": "postcss src/index.css -o public/bundle.css",
"build:md": "sitedown src -b public -l src/layout.html",
"build:feed": "generate-feed src/log --dest public && cp public/feed.xml public/atom.xml",
"build:static": "cpx 'src/**/*.{png,svg,jpg,jpeg,pdf,mp4,mp3,js,json,gif}' public",
"build:icon": "gravatar-favicons --config favicon-config.js",
"watch": "npm run clean && run-p watch:* build:static",
"watch:css": "run-s 'build:css -- --watch'",
"watch:serve": "browser-sync start --server 'public' --files 'public'",
"watch:md": "npm run build:md -- -w",
"watch:feed": "run-s build:feed",
"watch:static": "npm run build:static -- --watch",
"watch:icon": "run-s build:icon",
"clean": "rimraf public && mkdirp public",
"start": "npm run watch"
}
}
- I have
postcss
building css bundles, enabling an@import
based workflow for css, as well as providing various transforms I found useful. - The markdown is built with
sitedown
. - I wrote a tool to generate a RSS feed from markdown called
generate-feed
- I generate favicons from a gravatar identifier with
gravatar-favicons
. js
bundling could easily be added in here withesbuild
or rollup.- Steps are grouped into
build
andwatch
prefixes, and managed with npm-run-all2 which provides shortcuts to running these tasks in parallel.
I successfully implemented this pattern across 4-5 different websites I manged. It work beautifully. Some of the things I liked about it:
- β Tools could be swapped out easily.
- β I could experiment individual ideas on individual sites, without forcing them on every other project.
- β All of the tools were versioned gated, and I could update them as I had time.
- β It worked very well with release automation.
But it had a few drawbacks:
- β I liked the pattern so much, that I wanted to add websites to more projects, but setting this all up was a hassle.
- β When I discovered a change I liked, re-implementing the change across multiple projects become tedious.
- β οΈ Orthogonal tool composition helps keep this arrangement clean and fast, but some steps really do benefit from the awareness of the results of other steps.
So I decided to roll up all of the common patterns into a single tool that included the discoveries of this process.
@siteup/cli
extended sitedown
Because it was clear sitedown
provided the core structure of this pattern (making the website part), I extended the idea in the project @siteup/cli
. Here are some of the initial features that project shipped with:
- Variable cascade + frontmatter: You could define global and page variables used to customize parts of the site layout. This lets you set the title, or other parts of the
<head>
tag easily from the frontmatter section of markdown pages. - A
js
layout: This let you write a simple JS program that receives the variable cascade and child content of the page as a function argument, and return the contents of the page after applying any kind of template tool available in JS. html
pages: CommonMark supportshtml
in markdown, but it also has some funny rules that makes it more picky about how thehtml
is used. I wanted a way to access the full power ofhtml
without the overhead of markdown, and this page type unlocked that.js
pages: Since I enjoy writing JavaScript, I also wanted a way to support generating pages using any templating system and data source imaginable so I also added support to generate pages from the return value of a JS function.- JavaScript bundling: I wanted to make it super easy to include client side JavaScript, so it included some conventions for setting that up quickly and easily.
- CSS bundling: Instead of adding the complexity of css-modules or other transform-based scoped css solution,
siteup
opted to make it super easy to add a globalcss
bundle, and page scopedcss
bundles, which both supported apostcss
@import
workflow and provided a few common transforms to make working withcss
tolerable (nesting, prefixing etc). - Static files: Including static files was such a common pattern that I also included the ability, to copy over static files without any additional configuration. I believe
sitedown
also added this feature subsequently. - Asset co-location: I wanted a way to co-locate assets, code and anything near where it was depended upon. It is such a natural way to organize complex collections, and so many tools break this, or require special locations for special files. Let me put it wherever I want in
src
!
After sitting on the idea of siteup
for over a year, by the time I published it to npm
, the name was taken, so I used the npm org name hack to get a similar name @siteup/cli
. SILLY NPM!
I enjoyed how @siteup/cli
came out, and have been using it for 2 year now. Thank you of course to ungoldman for laying the foundation of most of these tools and patterns. Onward and upward to top-bun
!