19 April 2019
Nikku is my latest side project that I’ve been working on since early April 2019. It is a BRSTM player on the web.
BRSTM is a file format that contains lossless audio that is usually used for some models of Nintendo consoles, and it’s pretty popular on the internet as people have reverse engineered to recreate those file format. I think that it’s popular because that this file format could encode a loop point in it.
Background
My encounter of this file was back in 2016, when I was still in university, preparing for exams, and naturally I sought out for some “study music”. I do have some game music that’s pretty nice to play on loop (Kingdom Hearts music), but that time, I was curious if there is something I could do to make it loop continuously, just like when that music is played in-game. After some googling, I found the website “Smash Custom Music” (formerly “Brawl Custom Music”) and I was amazed that it contained so much game music in it! The best thing is that, they let the visitors to download them. So, naturally, I went to the Kingdom Hearts page, and selected a music, but then when I was presented with a list of unfamiliar file formats. Since I was not familiar with any of them, I downloaded the first one on the list, which is BRSTM.
Naturally, I googled “how to play BRSTM file”, and I found BrawlBox. I think this is a program that could open/edit many file formats usually found in Nintendo consoles. One of the functionality of the program is … to play BRSTM file, with loop, and it works so well.
Until I finished my study and went to work. At work, I was issued a Macbook and that BrawlBox program isn’t supported out-of-box on macOS. But I still have a workaround: the website “Smash Custom Music” has a Flash widget that could play the audio on the site itself. But the clock is ticking, since Flash is being deprecated and browser will soon drop support for Flash plugin.
So, this idea of playing BRSTM file on the web has been around for a while, but I haven’t acted on it due to … various reasons (ehm, mostly laziness).
Dawn of Nikku
On early April 2019, I started to think about this harder. At first, my goal was just to write a Node server that could stream the BRSTM file. The very initial prototype is just a program that converts BRSTM file to MP3 using ffmpeg, and stream that MP3; and loop point could be read using the “ffprobe” program that comes along ffmpeg. Digging deeper, I read a lot about “live streaming audio” on the web, with “Media Source Extension” (note that most of the articles are about live streaming video and audio but for my use case, I only need the audio part). At this point, if I want to play the BRSTM file, I have to:
- start a Node program; this program will read and convert BRSTM file to MP3 and stream that on web server
- open browser to listen to that stream
I thought if I had over-engineer something that should be simpler.
Project Reset
I took a step back and scrapped everything.
I decided that my use case should be like this:
- User opens a web page
- User uploads BRSTM file
- User controls the playback
At this point, the real work has just been started.
I found that modern browser can read an uploaded file using FileReader API. The feeling of using this API is that it’s pretty old: it’s event callback-based instead of Promise-based. After the file is read, I get an ArrayBuffer. I converted this ArrayBuffer to Uint8Array and sliced the appropriate locations to retrieve metadata. I relied on this BRSTM file description on Wiibrew heavily to do so.
In short, BRSTM file consists of a header and 3 chunks:
- HEAD chunk, which consists of 3 parts and has lots of metadata in it.
- ADPC chunk, which I’m not really sure what it does
- DATA chunk, which is said to contain the audio data in “ADPCM” format.
After messing around with reading the file using FileReader API, now I have to wonder on how to play it on the web.
First Attempt: Naively rely on Web Audio API
So, naturally, I found Web Audio API’s decodeAudioData. So I sliced the BRSTM’s DATA chunk and feed it to decodeAudioData. Instead of getting a sound, I got an error. Oh well, at least I tried.
Second Attempt: NPM Package
I haven’t lost hope. Since browser don’t know how to decode “ADPCM” audio, I naturally assumed someone else have NPM package for that, and I was right!
I found this “imaadpcm” encoder/decoder package on NPM and tried to use it. That package is very well documented and even supports importing it directly in ES2015 module script right in browser without any “build” step.
Naively, I sliced the DATA chunk out from the BRSTM as Uint8Array, pass it to the decode function, alongside the “blockSize” that I read from BRSTM metadata, and I got back a Int16Array “PCM samples”.
The final step is to convert that PCM samples (Int16Array) into what Web Audio API supports (Float32Array). This step is basically interpolation from Int16 to Float32.
At this point, it played a sound, but… it’s full of noises, but if I listen closely, I could hear the real music itself, covered with lots and lots of noise. Plotting the samples out, I can really see that it’s really noisy. The sample results just varied wildly
This got me thinking, maybe I decoded it wrongly, as I didn’t use that “ADPC chunk” at all, but then again, there is no place to input this chunk data (it is said to contain the coefficients for the decoder) in the “imaadpcm” decoder.
I’m now kind of lost on how to decode the file.
Third Attempt: BrawlBox!
So I was reminded of BrawlBox, which is open source! I cloned the repository, downloaded Visual Studio, build the project, and started to put debugger breakpoints to find out which files are responsible for decoding BRSTM. I found out the relevant codes are:
- RSTM.cs, acts as pointer to the various sections of BRSTM file
- ADPCMStream.cs, doing the actual reading of BRSTM file
- ADPCMState.cs, the actual decoding
After writing the decoder codes that’s largely adapted from those three files, I was still disappointed by the static noises when playing the BRSTM music. This time, the static noise was even louder than before (and the true music wasn’t even audible anymore).
But then, I realized, I didn’t follow the exact codes from BrawlLib (typos here and there). After I corrected it, IT WORKS! THE MUSIC WAS SO CLEAR! 😂😂😂😂😂😂
After that moment, the rest of the work are just to clean up the codes, put up a simple UI, figure out how to do looping using Web Audio API, and finally open source the codes!
Here We Are
Check out the demo and the codes at the links below!
Anyway, I’m not done with this yet. In the near future, I shall work on the following items:
- Improve the looping (every time the audio loops, an annoying “gap” is heard,
as far as I know, this is due to how Web Audio API is designed)- update 2019-04-24: seems like it might not be due to Web Audio API but due to my decoder
- update 2019-04-25: turns out to be a bug in my BRSTM decoder. I’ve fixed this and looping is now gapless!
- Improve the UI: add controllable progress bar
- Run BRSTM decoder in web worker context
- update 2019-04-24: unlikely to happen soon, since ES2015 modules are not supported in worker context
- Publish the BRSTM decoder as NPM package
- update 2019-04-24: done
About the Name
Oh, about the name “Nikku”. It was inspired from the character name Neeku from the TV series Star Wars Resistance. To me, I feel that this character is very talkative and couldn’t stop talking, hence I use it as the name for this “audio player”