Melody Maker

TLDR:  I struggled to get magenta js to do much. I created the most basic example of using the music VAE + the web midi api, separately. It uses the “mel_4bar_small_q” model which is “A 4-bar, 90-class onehot melody model. Less accurate, but smaller in size than full model. Quantized to 2-byte weights.”

For the midterm, I decided to experiment with Magenta. I dug around and discovered magenta.js, and decided to use it as I figured it would be simpler to set up than the regular python version. I was wrong because javascript, typescript and ES6+, are annoying.

The most basic documentation for magenta.js can be found in the announcement blog post. There are some example apps here and a list of technical examples here but the full documentation is generally obtuse.

I wanted to create something that would allow me to spit out random generated sequences to my music hardware. I got going with the web midi api, but it took a while to discover the proper documentation: http://webaudio.github.io/web-midi-api/

Ok so now I got my midi ports recognized and I could send notes from the browser to ableton using the internal midi bus…. Then I discovered this example which generates random sequences and outputs them over midi, exactly like I was hoping to do… I tried it out with my hardware synthesizer and it gets some nice results.

But at this point I’d gone too far to quit. And that example was hard to understand, as it used the main tensorflow library, and it was written in typescript… the model implementation isn’t so obtuse but I couldn’t figure out the midi parsing. Because javascript sucks at managing the timing of sequences…so now some more digging.

Magenta JS uses Tone JS for it’s basic ‘player’ functions that allow it to iterate over a note sequence. That player can be given a callback object, with run and stop methods… but I struggled to implement it properly. Here’s the code describing how the magenta player is implemented.  Unfortunately for me, the callback wouldn’t work.

In addition, I tried passing the note sequence from the VAE to the RNN and that didn’t work. Neither did converting that note sequence to a midi file.

here midi conversion works in an example
but in mine it just breaks…

 

Anyways, after all this frustration, I gave up. Here’s what I got.

–Update–

Got a response from some people at google. The documentation did not have the most recent version of magenta js referenced on their cdn. So now with <script src=”https://cdn.jsdelivr.net/npm/@magenta/music@1.1.2″> </script> the callbacks and basic midi conversion now works. Still would have to do more to handle the actual midi output.

Leave a Reply