Littlstar Portfolio

After a short sabbatical in Denmark at Hyperdivision, I joined Littlstar. Here is a quick overview of some of the more interesting projects I worked on. I joined Littlstar during a transitional period of the company and help them transition from a VR video platform to a video on demand and live streaming platform, and now an NFT sales and auction platform.

NYCPF VR Platform

NYCPF-XR screenshot
NYCPF used a local process talking over websocket RPC to a webapp running in a local browser.

My first project was picking up on an agency style project developing a novel VR reality training platform for NYCPF, powered by a custom hypercore p2p file sharing network, delivering in-house developed unity VR scenarios. These scenarios could then be brought to various locations like schools or events where NYCPF could guide participants through various law enforcement scenarios with different outcomes based on participant choices within the simulation.

By utilizing a p2p and offline first design approach, we were able to deliver an incredibly flexible and robust delivery platform that had all sorts of difficult features to develop for traditional file distribution platforms such as:

While the project was built on the backs of giants like the Hypercore protocol, as well as the amazing work of my colleague, I contributed in a number of areas to move the project forward during my time contracting on it.

Tiny Module Compiler

Some of the discrete software packages that resulted from this project are described below.

secure-rpc-protocol

Secure rpc-protocol over any duplex socket using noise-protocol. This was a refactor of an existing RPC over websocket solution the project was using. It improved upon the previous secure RPC already used by switching to using the noise protocol which implements well understood handshake patterns that can be shared and audited between projects, rather than relying on a novel implementation at the RPC layer. It also decoupled the RPC protocol from the underlying socket being used, so that the RPC system could be used over any other channels we might want in the future.

Secure RPC Protocol
Secure RPC protocol uses the noise protocol for encryption, and works over any duplex socket.

async-folder-walker

An async generator that walks files. This project was a refactor of an existing project called folder-walker implementing a high performance folder walk algorithm using a more modern async generator API.

Async folder walker
Async folder walker provides a modern api to folder and filer walking of a directory.

unpacker-with-progress

unpacker-with-progress is a specialized package that unpacks archives, and provides a progress api in order to provide UI feedback. One of the deficiencies with the NYCPF project when I started was lack of UI feedback during the extraction process.

VR files are very large, and are delivered compressed to the clients. After the download is complete, the next step to processing the data is unpacking the content. This step did not provide any sort of progress feedback to the user because the underlying unpacking libraries did not expose this information, or only exposed some of the information needed to display a progress bar.

This library implemented support for unpacking the various archive formats the project required, and also added an API providing uniform unpacking progress info that could be used in the UI during unpacking tasks.

Unpacker with progress
unpacker-with-progress is a consistency layer on top of a few unpacking libraries, with an emphasis on progress reporting for use in UI applications.

hchacha20

One of the more interesting side projects I worked on was porting over some of the libsodium primitives to sodium-javascript. I utilized a technique I learned about at Hyperdivision where one can write web assembly by hand in the WAT format, providing a much wider set of data types, providing the type guarantees needed to write effective crypto.

While the WAT was written for HChaCha20, the effort was quite laborious and it kicked off a debate as to whether it would be better to just wrap libsodium-js (the official libsodium js port) in a wrapper that provided the sodium-universal API. This was achieved by another group in geut/sodium-javascript-plus which successfully ran hypercores in the browser using that wrapper.

Ultimately, this effort was scrapped, determining that noise peer connections in the browser are redundant to webRTC encryption and https sockets. It was a fun and interesting project none the less.

Some HChacha WAT Code
A peek into the world of WASM via WAT.

Reconnecting sockets

We were having some state transition bugs between the webapp and the local server process, where the app could get into strange indeterminate states. Both had independent reconnect logic wrapped up with application code, and it added a lot of chaos to understanding how each process was behaving when things went wrong (especially around sleep cycles on machines).

I implemented a generic reconnecting state machine that could accept any type of socket, and we were able to reduce the number of state transition bugs we had.

Reconnecting simple websocket

Little Core Labs

After working on various agency projects at Littlstar, we formed a separate organization to start a fresh rewrite of the technology stack.

My high level contributions:

This was an amazing founders-style opportunity to help rethink and re-implement years of work that had developed at Littlstar prior to joining. Effectively starting from 0, we rethought the entire technology pipeline, from operations, to infrastructure, to deployment, resulting in something really nice, modern, minimal, low maintenance and malleable.

Terraform ops

A culmination of ingesting the “Terraform: Up & Running” and “AWS Certified Solutions Architect” books, as well as building off existing organizational experience with AWS, I helped research and design an operations plan using Terraform and GitHub Actions.

This arrangement has proven powerful and flexible. While it isn’t perfect, it has been effective and reliable and cheap, despite its imperfections in relation to some of the more esoteric edge cases of Terraform.

A quick overview of how its arrange:

Github actions

Actions logo

One of the drawbacks of rolling our own Terraform CI infrastructure was that we had to tackle many small edge cases inside the GitHub actions environment.

It was nice to learn about the various types of custom GitHub actions one can write, as well as expand that knowlege to the rest of the org, but it also ate up a number of days focusing on DevOps problems specific to our CI environment.

Here are some of the problems I helped solve in the actions environment.

netrc-creds

sdk-js

I helped lay the framework for the initial version of sdk-js, the Little Core Labs unified library used to talk to the various back-end services at Little Core Labs.

One of underlying design goals was to solve for the newly introduced native ESM features in node, in such a way that the package could be consumed directly in the browser, natively as ESM in node, but also work in dual CJS/ESM environments like Next.js. While this did add some extra overhead to the project, it serves as a design patter we can pull from in the future, as well as a provide a highly compatible but modern API client.

I also extracted out this dual package pattern into a reusable template.

sdk-js

rad.live

Rad.live

I was the principal engineer on the new Rad.live website. I established and implemented the tech stack, aiming to take a relatively conservative take on a code base that would maximize the readability and clarity for a Dev team that would eventually grow in size.

A high level, the application is simply an app written with:

From a development perspective, it was important to have testing and automation from the beginning. For this we used:

Overall, the codebase has had two other seasoned developers (one familiar and one new at React) jump in and found it productive. Additionally, it has been a relatively malleable code base that is easy to add MVP features to and is in a great position to grow.

vision.css

vision.css

I implemented a custom design system working closely with our design team. This project has worked out well, and has so far avoided ‘css lockout’, where only one developer can effectively make dramatic changes to an app layout due to an undefined and overly general orthogonal ‘global style sheet’.

The way this was achieved was by focusing on a simple global CSS style sheet that implements the base HTML elements in accordance with the design system created by the design team. While this does result in a few element variants that are based on a global style class name, they remain in the theme of only styling ‘built in’ html elements, so there is little question what might be found in the global style sheet, and what needs to be a scoped css style.

Some of the features we used for vision.css

eslint-config-12core

eslint-config-12core

Linting is the “spell check” of code, but its hard to agree on what rules to follow. Like most things, having a standard set of rules that is good-enough is always better than no rules, and usually better than an unpredictable and unmaintainable collection of project-unique rules.

I put together a shared ESLint config that was flexible enough for most projects so far at Little Core Labs based on the ‘StandardJS’ ruleset, but remained flexible enough to modify unique org requirements. Additionally, I’ve implemented it across many of the projects in the Github Org.

gqlr

gqlr

gqlr is a simplified fork of graphql-request.

This relatively simple wrapper around the JS fetch API has a gaggle of upstream maintainers with various needs that don’t really match our needs.

The fork simplified and reduced code redundancy, improved maintainability through automation, fixed bugs and weird edge cases and dramatically improved errors and error handling at the correct level of the tech stack. These changes would have unlikely been accepted upstream, so by forking we are able to gain the value out of open source resources, while still being able to finely tune them for our needs, as well as offer those changes back to the world.

local-storage-proxy

A configuration solution that allows for persistent overrides stored in local storage, including cache busting capabilities. Implemented with a recursive JS proxy to simulate native object interactions over a window.localstorage interface.

Community maintenance

We ended up taking on maintainence of a few other packages, providing fixes and improvements where the original authors seem to have left off.

Video platform

Here are some snapshots of the video platform we launched.

NFT Auction Platform

Here are some screenshots of the NFT auction platform I helped build. The UI was fully responsive and updated on the fly to new results, thanks to the powers of SWR.

Marketing pages

I did a few marketing pages as well.

Conclusion

While this isn’t everything I did at Littlstar, it captures many of the projects I enjoyed working on, and can hopefully provide some insights into my skills, interests and experiences from the past year.