You can also listen to the transcript of four Russian composers, including Rachmaninoff, playing this pattern recognition and generation game at a dinner party in the late 1800’s: https://youtu.be/PlFPOWuwBHI?is=EKBK7QQkJs4MsTCU
Composers at the time could do this just by looking at sheet music and audiating, without using a piano.
vunderba 1 days ago [-]
To add to this: classical extemporization, post-Romantic period, also sort of fell out of favor as the composer began to pre-empt the performer and the "written note is sacrosanct" effectively became the de jure standard.
There's a great story around how Beethoven, perhaps one of the strongest improvisers of his day, completely upstaged Steibelt, a contemporary musician and by all accounts a bit of a charlatan. I'll include the entire quote verbatim from "The Lives of the Great Pianists" by Harold Schonberg.
This time Steibelt had prepared a brilliant fantasy for piano and strings, the theme coming from the Beethoven trio he had heard the previous week. Steibelt's admirers were in raptures. Now the issue was joined, and Beethoven had to show his strength. He walked to the piano, grasped the cello part of the Steibelt work en route, put it upside down on the piano and insultingly drummed out a theme with one finger. Then he improvised, and—angry, excited, on his mettle—how he must have improvised! Before Beethoven had finished, Steibelt stole from the room. He never again would meet Beethoven and made it a condition before going anywhere in Vienna that Beethoven not be invited.
argimenes 22 hours ago [-]
Probably the last truly great classical improviser on piano was Anton Rubinstein (1829-1894). His Fantaisie in E minor, Op. 77 is probably the pinnacle of his skill in (notated) improvisation.
After that, the great classical improvisers were probably organists in the 20th century, like Marcel Dupré (1886-1971) and Pierre Cochereau (1924-1984). Cochereau created vast, highly personal, symphonic forms with dazzling orchestral registration.
dyzone 1 days ago [-]
I'm a little lost. What am I listening for in that video?
tom_vidal 1 days ago [-]
Check the article for the full context. It’s basically a musical version of exquisite corpse. One composer wrote the start of a phrase (on sheet music, without using a piano), then passed it on to the next composer, who continued the phrase based only on reading the sheet music.
This illustrates the extraordinary degree of audiation, or the cognitive ability to imagine music in the mind in a symbol-linked, systematized way, that used to exist in classical music.
Each composer could read what was just written, audiate how it would sound, audiate a continuation of the phrase, and then transcribe that aural imagination back to sheet music.
hiccuphippo 1 days ago [-]
Classical jamming.
jasonjmcghee 1 days ago [-]
I think this is a great project and very HN. Not sure why the comments are so focused on the deliverable- you learned way more and had a much more interesting experience.
One think I didn't see mentioned in the post- maybe I missed it- how large was the data? How many samples did you use to pretrain and post-train
munksbeer 1 days ago [-]
There is this in the post:
> The final dataset contained a few hundred thousand MIDI files, representing roughly 300 million note events.
joshuamerrill 1 days ago [-]
Classical pianist and software product designer here.
I see so much in common with this project and the numerous AI-based UX design tools out there. Whether it's music or UI, now that the "generation" portion of the work costs zero, all that remains is taste.
And so much of taste comes from exploring and killing off possibilities that turn out to be dead-ends. I love the idea that models like these will help us find the dead ends faster, or even produce a gem here and there.
P.S. if you want another uncanny version of Fur Elise, listen to Beethoven's own 1822 revision: https://www.youtube.com/watch?v=s24TtiGgb6k. His 1810 version that we all know was simpler and more balanced. But for what it's worth, Beethoven didn't publish either of them.
nuancebydefault 1 days ago [-]
Thanks. I found the 1822 version actually quite agreable
stickfigure 24 hours ago [-]
Interesting! It reminds me of something I once heard another banjo picker say... something like "I spent most of my youth figuring out how to cram more notes into a piece, and most of my maturity trying to remove them".
I am not fond of the extra jazzy notes.
bubblegumcrisis 1 days ago [-]
[flagged]
goda90 1 days ago [-]
Reminds me of this project to generate every melody possible algorithmically in order to fight music copyright lawsuits.
https://allthemusic.info/
ronyeh 1 days ago [-]
Thanks. I enjoyed watching the TEDx talk on that page.
karmelapple 1 days ago [-]
Hearing the start of Für Elise, and then it being taken in an incredibly different direction, is surprisingly disconcerting.
gregsadetsky 1 days ago [-]
There's an incredibly funny bit (if you're into absurdist comedy) by Hans Teeuwen about the disconcerting direction a well known song can take:
The disconcerting effect is enhanced by hearing the Dutch voice and reading the Dutch subtitles which don't quite match up with what he is saying in English! - until it all goes off into musical absurdity. Thanks for sharing.
ABNW 1 days ago [-]
I found it refreshing and interesting, not really disconcerting. Show's how AI can extrapolate in novel ways. An interesting feature.
1 days ago [-]
h2aichat 1 days ago [-]
Can we hear it?
senbrow 1 days ago [-]
It's on the linked page if you scroll down just a bit.
Interesting. I do feel letting a machine generating notes is taking the joy out of improvisation, is it not?
I think one of the great, early, joys of learning a piano is gaining the following intuitions: The seemingly harder path of learning sheet is actually faster. Your mind _should_ learn to think in two dimensions Spatial — where fingers go — and Time — pitch and tempo – ** when learning. The _internalization_ of Space and Time queues guide the fingers in a dance that is vastly satisfying. This skill leads you to the final part of the journey that is improvisation and the one more exciting than what i am on now.
---
**
Space: Your finger placement on keys right, e.g. knowing how to go from landmark/anchor notes(mid-c, G, F etc) and then go to the others above and below it. Crudely this is some what like typing from your landmark f and j qwerty keyboard
Time: The out singing/verbalizing of the notes/beats on a time measure as you play them(per the time measure). e.g. you can say out loud 1-2-3-4 for 4/4 measure, if the measure has quarter notes say out loud. And `1-e-and-a-2-e-and-a-3-e-and-a-4-e-and-a` for a 4/4 with 1/16th note granularity. Do this as you play the notes and you get a sense of tempo.
elwell 1 days ago [-]
OP found a different source of joy: "I wanted the fun of working through the problem myself, rather than just implementing someone else’s research."
ghm2199 1 days ago [-]
Fair enough.
jeffreygoesto 1 days ago [-]
Reminds me of a story a coworker told me about his band rehearsal when the new drummer counted one-two-three-four-five-six-se-ven-eight.
yieldcrv 1 days ago [-]
as with every discipline that AI and therefore computers can reason through suddenly:
you could always do the discipline you liked before for self fulfillment purposes, and you can still do it for self fulfillment purposes now
debugnik 1 days ago [-]
But most people who want to get good at their main discipline, even for self-fulfillment purposes, have been gaining the necessary experience through their job, because an unrelated job takes away too much time to do so.
yieldcrv 1 days ago [-]
its a good motivator and looks like we'll be returning to the more feudal version where just that class was focusing on the arts, only because everyone else is preoccupied with doing the things that pays the bills
unless they're actually serfs again and won't have bills to pay, but no access to anything outside of the fiefdom that they maintain all day
heikkilevanto 1 days ago [-]
I would love something like that, except that I play the melody, and it produces proper 3-4 part accompaniment, preferably in good baroque style. Extra bonus if it could also write it into a file in a format suitable for music editing programs.
vunderba 1 days ago [-]
An early attempt at this was Microsoft Songsmith [1] all the way back in 2009, which would take a melody (usually recorded by mic) and try to scaffold an accompaniment around it though obviously not realtime in any sense of the word.
The closest we've had to realtime orchestration around a melody in the "real world" is probably arranger keyboards though your left hand is still responsible for the chord progression itself.
Did you ever try Ludwig? It was rule-based by the Fritz chess engine maker. To my understanding Ludwig was grounded in traditional harmony, counterpoint, voice-leading, and orchestration principles taught in formal music education. It never took off so they eventually stopped it.
I am still waiting for something like this based on generaitive AI.
Naitik88 1 days ago [-]
This feels like a natural next step. Starting with a simple melody and having the system fill in the rest while still following your playing style could make it much more useful for experimentation than generating a complete piece from scratch.
morkalork 1 days ago [-]
That's a fun idea. You could start playing the piano and it kicks in with a base and drums for a jazz band.
altmanaltman 1 days ago [-]
That is an incredibly hard challenge though. Creating the whole backing track (in any meaningful way other than just basic chords) from just melody will require an amazingly high number of highly subjective choices and random gen will not lead to good outcomes since our ears like intentional and artistical creativity in general.
jancsika 20 hours ago [-]
The end of the AI's first sub-phrase in the video is wrong.
You started by playing a simple I - vii - I as the first sub-phrase. Then the AI started its response with a dominant 7th-chord. In the vast majority of cases I can think of from the Classical era, the AI's sub-phrase would end with a half-cadence. So, including what you played, the first two sub-phrases would look like this:
I - vii - I
V(7) - I - V
But instead, the AI took the unusual step of having a full cadence for that 2nd sub-phrase. This in music is what someone might call a "double beat" in screenwriting.
Moreover, there are no half cadences in the phrase after that. It's all full cadences in the tonic key, plus two plagal cadences over a tonic pedal there in the 2nd phrase. Each sub-phrase just extends the tonic. There are idiomatic ways to just stay in the tonic for awhile, like "Horn fifths." But this excerpt isn't idiomatic and sounds like the composer accidentally forgot to develop the harmony.
I'm going to claim that this excerpt would sound like the musical equivalent of a run-on sentence to any composer who lived from the time of J.S. Bach to Robert Schumann. And from Mahler on they'd assume you're thumbing your nose at convention.
Put another way: the kind of student who'd use this to finish a harmony exercise would end up getting the same grade they would have gotten doing it on their own.
Edit: clarifications
hliyan 1 days ago [-]
Ah, MIDI files. The only type of music you could realistically download from the internet back in the day, and you had to wake up at ungodly hours so that your dialup modem would not rack up a massive phone bill.
killerstorm 1 days ago [-]
MIDI is still widely used for professional music production. It sounded goofy back in the day because synthesizer it was played on was not very good.
andai 1 days ago [-]
I remember at one point RuneScape switched from the built in Microsoft midi whatever to their own sound engine, and from that day, everything sounded wrong, even the frogs, because to me, the crappy midi sounds were the whole personality and feel of the game.
HPsquared 1 days ago [-]
I'm thinking canyon.mid on Microsoft GS Wavetable Synth.
1 days ago [-]
otabdeveloper4 1 days ago [-]
MIDI is a protocol, and it's not ever going away or being replaced.
Some protocols (like I2C or MIDI) are gonna be with humanity forever, probably.
xtracto 1 days ago [-]
I remember downloading MOD files. It was just a bit larger than midi, but sounded better. I had a PC but I think it was an Amiga thing
andai 1 days ago [-]
Need For Madness is one of my favorite games, and probably two thirds of that fondness is the music, which was a bunch of MOD files.
rajivayyangar 1 days ago [-]
The results strike me as comparable or worse than you could get with a Markov model. I think it reveals the gap in understanding between LLMs and music. I think you need to either:
- Set up a pipeline to decompose music into, say, harmonic sequences and melodic sequences, and then have the LLM work on some more fundamental or more high-level layer of musical composition and then re-translate it back into actual sounds.
- Develop a better dataset and train the LLM more natively on musical examples.
Does anybody know of a project that has produced more convincing results?
shannifin 21 hours ago [-]
I'm not at all there yet, but this sort of thing is on my roadmap for my AI music project at tunesage.com ... That is, generating musical ideas while being conscious of phrase / melodic / thematic structure that a user can then play around with.
While LLMs (and ANNs in general) are definitely great for musical analysis (though still limited), I think they're likely overkill for the generation part, plus they tend to suffer from "creative collapse" as they prefer averages rather than exploration (which can be good or bad depending on what you want).
Computer0 1 days ago [-]
There is no LLM in the ML pipeline provided by the OP.
devonsolomon 1 days ago [-]
This is really fun. Scaler 3 starts with a chord progression and lets you break it down into musical performances and parts. Useful for ideation when producing.
Would be fun to get a midi clock going and play some chords on my piano and have my synth start jamming along with the bass and my keyboard doing some performance. Or any combination of the above.
mh- 1 days ago [-]
+1 all of this. That would be incredible (this already seems very cool - excited to get home and try it!)
Naitik88 1 days ago [-]
That would be a really interesting direction. At that point it starts feeling less like autocomplete and more like having another musician reacting to what you're playing in real time.
Tepix 1 days ago [-]
Oh, a cool idea! I just tried it, works pretty well. Kudos!
One feature request:
Instead of playing the AI-generated audio solely through the iPhone's speakers, add an option to send the audio as midi notes to a device (probably the same one you received the mini notes from).
evalystai 1 days ago [-]
The idea is awesome! :) However there's definitely much room for improvement, first of all rythm and composition (so there's some sense of musical form).
simedw 1 days ago [-]
Thank you.
Yes, I think I’ve gotten it to roughly a GPT-2 level: good enough to share, but with a lot of room left to improve. I think adding some kind of bar/measure token might help with rhythm, and perhaps some form of longer-term planning for the overall composition.
Also consider checking their decisions about representation, etc
pil0u 1 days ago [-]
Even after a few years deep into AI, I find your application absolutely magic. This is very inspiring, thank you for sharing.
sagebird 1 days ago [-]
Für Elsevier Journal Access Library pass out candy for safety this halloween your baby off milk shake it off.
elwell 1 days ago [-]
I, für one, welcome our new LLM overlords.
isoprophlex 1 days ago [-]
Amazing idea! Gonna hook this up to my little synthesizer and blast some square wave arpeggiated ML music!
davidajackson 1 days ago [-]
How would you expand this to support elements like attack ("velocity of the key-down" in piano speak), grace notes, timing etc. Would each of those be part of this model or another model? How would you model an arbitrary element (pedal, duration, etc...)
simedw 1 days ago [-]
If the attribute describes the current note, I would first try adding it as another field/head on the note event. For example:
The harder part might actually be finding enough good training data with all of those attributes represented consistently.
Ohentis 1 days ago [-]
It occurres to me that the relationship between pitches usually matters more than the pitches themselves. Perhaps the pitches could be encoded in a vector like what is typically done for position.
ingvay7 18 hours ago [-]
This is amazing. Great inspiration to train a model for a midi electric guitar solo. Though with bends, vibratos, mutes, harmonics, dive bombs etc as part of the event feed, it may possibly create way more events than this which means more training I suppose.
subhajeet2107 1 days ago [-]
This is so amazing, can you improve the quality of generation at the cost of notes per seconds ? No one can play 108 notes/sec anyways, maybe you can train the model to do CoT for better quality
simedw 1 days ago [-]
Yes, some kind of planning step is on my TODO list. Another thing I want to try is generating a few continuations in parallel, picking the one that looks best, and then continuing from there. Maybe the picking could be automatic.
I can probably squeeze out quite a bit more than 100 notes/sec as well. I haven’t spent much time optimizing inference yet.
butlike 1 days ago [-]
Question as a musician: Do you find satisfaction in the piece it created in-and-of-itself (subtracting the satisfaction from making the tool itself)?
gaya3bollineni 1 days ago [-]
Running a 125M model on-device at that speed is impressive. How much did you have to optimize the model to get that performance on an iPhone?
simedw 1 days ago [-]
The biggest speed improvement came from changing the note representation when I switched to compound note events: roughly 5× fewer autoregressive passes per note.
For the current model I’m using Core ML, which optimizes the kernels the first time you run it. I haven’t actually spent that much time tuning performance beyond that.
Naitik88 1 days ago [-]
The answer about changing the note representation was interesting. Sometimes a change in how the problem is represented ends up giving a much bigger improvement than trying to optimize the model itself.
ryanschneider 1 days ago [-]
> I trained a 125M model to autocomplete piano on-device
I first parsed the "on device" as _on the piano_ and was intrigued how that worked :) maybe someone can make "dynamic" rolls for a player piano somehow (like one of those braille "screens")?
leobg 1 days ago [-]
I don’t have MIDI. How about whistling or playing the piano via microphone? Sounds easy. Another 6 month rabbit hole? :)
speedgoose 1 days ago [-]
Talking about AI music with some live human MIDI inputs, Magenta Realtime 2 was released a few weeks ago and is pretty fun.
I have about 60 hours of my own playing in midi data. I wonder if I could fine-tune this model on that?
axoltl 1 days ago [-]
Looks neat! I'd love for it to feed the MIDI notes back into my player piano instead of playing out of my iPhone's (comparatively tinny) speakers though.
nl 21 hours ago [-]
This would be great to be able to use via Web Midi. That size LLM will run well in a browser.
Prompting and having the model fill out the rest makes it feel like jazz or at least an improvised result of a song
oleggromov 1 days ago [-]
This is almost exactly what Jordan Rudess is doing now with some folks from MIT or Stanford right now. Google his interview with Rick Beato.
jwr 1 days ago [-]
> Eventually I used Gemini 3.5 Flash for pairwise evaluation
But, but… wouldn't that be… (gasp) DISTILLATION?
Fun project!
Tostino 1 days ago [-]
The horror!
h2aichat 1 days ago [-]
Congratulations! It is amazing!!! Can you do the same with a song and give different drums to see which one fits better?
abidlabs 1 days ago [-]
Are the weights of the model available?
arkmm 1 days ago [-]
Very cool! Can you say a little bit about the size of the DPO training examples and how long training took?
simedw 1 days ago [-]
For DPO I only had around 700 preference examples, so not much data at all. That took about 12 minutes to train on a single GPU.
Pretraining was obviously a a lot slower, the 125M model took roughly half a day.
dandaka 1 days ago [-]
I would love to see a jamming partner. So I could play along him on the same piece.
yoouareperfect 1 days ago [-]
This is really awesome thanks for sharing
theschwa 1 days ago [-]
Would you be willing to share about how much it cost to train a model like this?
simedw 1 days ago [-]
Luckily I have access to 4x RTX 4090s, so I didn’t have to pay cloud GPU prices directly. If I had, it probably would have added up quite a bit given how many training runs and experiments I ended up doing.
1 days ago [-]
dyzone 1 days ago [-]
I wonder if training SOTA models on this will make them more humane?
lvbyte 1 days ago [-]
Any way to share the actual model? I don't have an iPhone to run the app on.
the_black_hand 1 days ago [-]
Cool work. I tried using LLMs to parse sheet music and they are really bad.
Naitik88 1 days ago [-]
[dead]
bilsbie 1 days ago [-]
Should tokens be multiple
Midi notes? Why or why not?
cillianpower 24 hours ago [-]
This would make a brilliant VST or Max 4 Live device if you would ever consider doing so.
luciana1u 1 days ago [-]
next step is a model that autocompletes the part where you actually practice. mine just sits there judging my scales.
1 days ago [-]
fl0id 13 hours ago [-]
Like... why.
noman-land 1 days ago [-]
This is honestly astonishing and the first "AI music" I've heard that has the potential to sound beautiful. I always thought that MIDI would be a perfect format for this. Glad to see this person make it happen!
aghilmort 1 days ago [-]
really incredible work! great use case, impeccable learning strategy, congrats!
amarbirsingh 1 days ago [-]
Pretty Impressive!!
mandeepj 1 days ago [-]
Gemma 4 E2B was too heavy for your needs?
micromacrofoot 1 days ago [-]
I too am currently learning to play Sonatina in G Major, it's a fun one! the end is surprisingly tricky
1 days ago [-]
kiops 6 hours ago [-]
[flagged]
leilakhanova 1 days ago [-]
[flagged]
1 days ago [-]
bubblegumcrisis 1 days ago [-]
[flagged]
jamilton 1 days ago [-]
>I guess no one actually wants to learn about harmony, about voicing, and voice leading, spend the hours. No one wants to learn how to actually play an instrument. I guess no one is willing to do the work. They want to just press some keys and declare that they made what the computer generated.
I don't understand that as a takeaway. Even if this worked perfectly, it would not be meaningfully stopping or discouraging anyone from learning music, and no one is claiming that what this models outputs is something the player played. It's a toy that someone made presumably because they like piano.
Ylpertnodi 1 days ago [-]
It's only moving air.
qmmmur 1 days ago [-]
[flagged]
kennyadam 1 days ago [-]
Is this HN? Aren't people supposed to tinker with tech for no reason other than seeing if they can? Is everything that involves a transformer now just AI BAD? Is that what the world has devolved into? Each side screaming "Orange Man Bad" and endless variations at each other?
meerita 1 days ago [-]
It is tiring to read these people. It feels like nothing will please them, but their own ideas, work, methods.
Forgeties79 1 days ago [-]
>orange man bad
You know, I was nodding along until you shoehorned that one in.
kennyadam 1 days ago [-]
[flagged]
Jtarii 1 days ago [-]
It's fairly obvious you are a trump supporter.
Forgeties79 1 days ago [-]
I’m plenty calm, I was just stating an objection. I don’t know where you’re getting the idea that I am not.
tescreal 1 days ago [-]
While it is a neat parlor trick, a lot of people have specific grevience against the application to art. AI has only served to further disempower artists broadly, and arguably it pushes "art" to a lower common denominator. Try to actually situate yourself in "why" people get rankled instead of making it a thought terminating cliche.
dgellow 1 days ago [-]
Given the AI crowd is very loudly telling us since years how humans will be replaced by LLMs and we will all be poor and left behind if we don’t join their cult, I think it’s a perfectly reasonable reaction. Anything with AI mixed with Art or other human experience is suspicious.
Also, yes, the orange fascist who attempted to coup his way to power, raped women, and is destroying democratic institutions is indeed bad.
phoghed 1 days ago [-]
No they haven’t. It’s like two guys that nobody really believes. You specifically seek that information out to get mad, then proceed to see it where it’s not. Like when someone makes a cool personal project about their hobby and it happens to involve AI.
dgellow 1 days ago [-]
I’m not mad and have nothing against the project.
> No they haven’t. It’s like two guys that nobody really believes.
It’s the entire leadership of the AI industry, in case you haven’t noticed
1 days ago [-]
Razengan 1 days ago [-]
> Given the AI crowd is very loudly telling us since years how humans will be replaced by LLMs and we will all be poor
Oh no, AI will take our jobs??? FUCKING LET IT!
Who even wants to do all these jobs if we don't HAVE to??
Ask politicians to give us UBI.
You want to attack the shit that could make shit easier instead of attacking the 200 year old institutions in place that ensure class divisions and perpetual debt and wage slavery? Smart buggers
dgellow 1 days ago [-]
> You want to attack the shit that makes shit easier instead of attacking the 200 year old institutions in place that ensure class divisions and perpetual debt and wage slavery
I have bad news for you if you think that will improve with the current deployment of AI. You will also notice I didn’t mention anything about jobs
perching_aix 1 days ago [-]
4) it's fun and quaint, like an odd, jazzy version of a player piano from an alternate universe
isoprophlex 1 days ago [-]
man hits thing
thing go plink
machine hear plink
machine make many more plink
man happy for plink is fun
man not hit machine with club or scream on orange site
man leave cave and touch plant
Razengan 1 days ago [-]
What is the goal here?
To eventually prove that Intelligence™ and Creativity™ isn't special?
Or maybe there is some other "spark" deeper within?
When everyone can make anything as soon as they think of it, what will set us apart?
For anyone interested, I’d highly recommend reading Robert Gjerdingen’s article Gebrauchs-Formulas. https://www.researchgate.net/publication/259731561_Gebrauchs...
You can also listen to the transcript of four Russian composers, including Rachmaninoff, playing this pattern recognition and generation game at a dinner party in the late 1800’s: https://youtu.be/PlFPOWuwBHI?is=EKBK7QQkJs4MsTCU
Composers at the time could do this just by looking at sheet music and audiating, without using a piano.
There's a great story around how Beethoven, perhaps one of the strongest improvisers of his day, completely upstaged Steibelt, a contemporary musician and by all accounts a bit of a charlatan. I'll include the entire quote verbatim from "The Lives of the Great Pianists" by Harold Schonberg.
After that, the great classical improvisers were probably organists in the 20th century, like Marcel Dupré (1886-1971) and Pierre Cochereau (1924-1984). Cochereau created vast, highly personal, symphonic forms with dazzling orchestral registration.
This illustrates the extraordinary degree of audiation, or the cognitive ability to imagine music in the mind in a symbol-linked, systematized way, that used to exist in classical music.
Each composer could read what was just written, audiate how it would sound, audiate a continuation of the phrase, and then transcribe that aural imagination back to sheet music.
One think I didn't see mentioned in the post- maybe I missed it- how large was the data? How many samples did you use to pretrain and post-train
> The final dataset contained a few hundred thousand MIDI files, representing roughly 300 million note events.
I see so much in common with this project and the numerous AI-based UX design tools out there. Whether it's music or UI, now that the "generation" portion of the work costs zero, all that remains is taste.
And so much of taste comes from exploring and killing off possibilities that turn out to be dead-ends. I love the idea that models like these will help us find the dead ends faster, or even produce a gem here and there.
P.S. if you want another uncanny version of Fur Elise, listen to Beethoven's own 1822 revision: https://www.youtube.com/watch?v=s24TtiGgb6k. His 1810 version that we all know was simpler and more balanced. But for what it's worth, Beethoven didn't publish either of them.
I am not fond of the extra jazzy notes.
https://www.youtube.com/watch?v=FCDPxNyLAww
If you like such creative AI work, I would recommend looking at the other "NeurIPS Creative AI Track 2025" submissions as well.
https://www.francoispachet.fr/continuator/
I think one of the great, early, joys of learning a piano is gaining the following intuitions: The seemingly harder path of learning sheet is actually faster. Your mind _should_ learn to think in two dimensions Spatial — where fingers go — and Time — pitch and tempo – ** when learning. The _internalization_ of Space and Time queues guide the fingers in a dance that is vastly satisfying. This skill leads you to the final part of the journey that is improvisation and the one more exciting than what i am on now.
---
**
Space: Your finger placement on keys right, e.g. knowing how to go from landmark/anchor notes(mid-c, G, F etc) and then go to the others above and below it. Crudely this is some what like typing from your landmark f and j qwerty keyboard
Time: The out singing/verbalizing of the notes/beats on a time measure as you play them(per the time measure). e.g. you can say out loud 1-2-3-4 for 4/4 measure, if the measure has quarter notes say out loud. And `1-e-and-a-2-e-and-a-3-e-and-a-4-e-and-a` for a 4/4 with 1/16th note granularity. Do this as you play the notes and you get a sense of tempo.
you could always do the discipline you liked before for self fulfillment purposes, and you can still do it for self fulfillment purposes now
unless they're actually serfs again and won't have bills to pay, but no access to anything outside of the fiefdom that they maintain all day
The closest we've had to realtime orchestration around a melody in the "real world" is probably arranger keyboards though your left hand is still responsible for the chord progression itself.
[1] - https://en.wikipedia.org/wiki/Microsoft_Research_Songsmith
Some details: https://shop.chessbase.com/en/products/ludwig_3_engl
You can still download it for free: https://www.heise.de/download/product/ludwig-58854 (Heise is a renowned German publisher, nothing shady)
I am still waiting for something like this based on generaitive AI.
You started by playing a simple I - vii - I as the first sub-phrase. Then the AI started its response with a dominant 7th-chord. In the vast majority of cases I can think of from the Classical era, the AI's sub-phrase would end with a half-cadence. So, including what you played, the first two sub-phrases would look like this:
I - vii - I
V(7) - I - V
But instead, the AI took the unusual step of having a full cadence for that 2nd sub-phrase. This in music is what someone might call a "double beat" in screenwriting.
Moreover, there are no half cadences in the phrase after that. It's all full cadences in the tonic key, plus two plagal cadences over a tonic pedal there in the 2nd phrase. Each sub-phrase just extends the tonic. There are idiomatic ways to just stay in the tonic for awhile, like "Horn fifths." But this excerpt isn't idiomatic and sounds like the composer accidentally forgot to develop the harmony.
I'm going to claim that this excerpt would sound like the musical equivalent of a run-on sentence to any composer who lived from the time of J.S. Bach to Robert Schumann. And from Mahler on they'd assume you're thumbing your nose at convention.
Put another way: the kind of student who'd use this to finish a harmony exercise would end up getting the same grade they would have gotten doing it on their own.
Edit: clarifications
Some protocols (like I2C or MIDI) are gonna be with humanity forever, probably.
Does anybody know of a project that has produced more convincing results?
While LLMs (and ANNs in general) are definitely great for musical analysis (though still limited), I think they're likely overkill for the generation part, plus they tend to suffer from "creative collapse" as they prefer averages rather than exploration (which can be good or bad depending on what you want).
Would be fun to get a midi clock going and play some chords on my piano and have my synth start jamming along with the bass and my keyboard doing some performance. Or any combination of the above.
One feature request:
Instead of playing the AI-generated audio solely through the iPhone's speakers, add an option to send the audio as midi notes to a device (probably the same one you received the mini notes from).
Yes, I think I’ve gotten it to roughly a GPT-2 level: good enough to share, but with a lot of room left to improve. I think adding some kind of bar/measure token might help with rhythm, and perhaps some form of longer-term planning for the overall composition.
Also consider checking their decisions about representation, etc
I can probably squeeze out quite a bit more than 100 notes/sec as well. I haven’t spent much time optimizing inference yet.
For the current model I’m using Core ML, which optimizes the kernels the first time you run it. I haven’t actually spent that much time tuning performance beyond that.
I first parsed the "on device" as _on the piano_ and was intrigued how that worked :) maybe someone can make "dynamic" rolls for a player piano somehow (like one of those braille "screens")?
https://magenta.withgoogle.com/magenta-realtime-2
But, but… wouldn't that be… (gasp) DISTILLATION?
Fun project!
Pretraining was obviously a a lot slower, the 125M model took roughly half a day.
I don't understand that as a takeaway. Even if this worked perfectly, it would not be meaningfully stopping or discouraging anyone from learning music, and no one is claiming that what this models outputs is something the player played. It's a toy that someone made presumably because they like piano.
You know, I was nodding along until you shoehorned that one in.
Also, yes, the orange fascist who attempted to coup his way to power, raped women, and is destroying democratic institutions is indeed bad.
> No they haven’t. It’s like two guys that nobody really believes.
It’s the entire leadership of the AI industry, in case you haven’t noticed
Oh no, AI will take our jobs??? FUCKING LET IT!
Who even wants to do all these jobs if we don't HAVE to??
Ask politicians to give us UBI.
You want to attack the shit that could make shit easier instead of attacking the 200 year old institutions in place that ensure class divisions and perpetual debt and wage slavery? Smart buggers
I have bad news for you if you think that will improve with the current deployment of AI. You will also notice I didn’t mention anything about jobs
thing go plink
machine hear plink
machine make many more plink
man happy for plink is fun
man not hit machine with club or scream on orange site
man leave cave and touch plant
Or maybe there is some other "spark" deeper within?
When everyone can make anything as soon as they think of it, what will set us apart?