https://thoughtstreams.io/jtauber/note-quantization/Note Quantizationjtauber2014-04-25T02:23:26Zhttps://thoughtstreams.io/jtauber/note-quantization/#card-4478There is another issue I need to addr...2014-04-25T02:23:26Z2014-04-25T02:22:53Z
There is another issue I need to address before finally getting to questions of how to actually infer a grid from a performance.
Imagine that we have a ritardando across two measures, m and m+1 such that T<sub>m</sub> = 100, T<sub>m+1</sub> = 102, T<sub>m+2</sub> = 104. In other words τ<sub>m</sub> = 2 and τ<sub>m+1</sub> = 4.
The tempo doesn't suddenly halve between measure m and measure m+1. We need to work out a decent model that adjusts each beat-group, beat and sub-beat τ appropriately for a continuous change in tempo.
https://thoughtstreams.io/jtauber/note-quantization/#card-4477I'm wondering now about the redundanc...2014-04-25T01:59:56Z2014-04-25T01:59:56Z
I'm wondering now about the redundancy in the fact that
> τ<sub>mgb1</sub> + τ<sub>mgb2</sub> = τ<sub>mgb</sub>
assuming S<sub>mgb</sub> = 2.
Related is the fact that any t ending in 1 (e.g. t<sub>1</sub>, t<sub>m1</sub>, t<sub>mg1</sub>, t<sub>mgb1</sub>) is always 0.
https://thoughtstreams.io/jtauber/note-quantization/#card-4472In the meantime, here's a diagram out...2014-04-25T01:55:23Z2014-04-25T00:50:31Z
In the meantime, here's a diagram outlining where we're currently at:
![note-quantization-2.png](/media/186/note-quantization-2.png)
https://thoughtstreams.io/jtauber/note-quantization/#card-4475What's particularly compelling about ...2014-04-25T01:32:34Z2014-04-25T01:17:51Z
What's particularly compelling about the above model for swing is that, as long as actual note placement is relative to the grid, we can easily swing a straight time rhythm or de-swing a swing rhythm back to a straight time just by changing the grid parameters τ<sub>mgb1</sub> and τ<sub>mgb2</sub>.
https://thoughtstreams.io/jtauber/note-quantization/#card-4474Swing can be modeled by shifting just...2014-04-25T01:15:41Z2014-04-25T01:15:41Z
Swing can be modeled by shifting just the even sub-beats. The following shows a single 4/4 measure without and with swing.
![swing.png](/media/185/swing.png)
Notice this can be modeled just as
> τ<sub>mgbs</sub> = 1/2 τ<sub>mgb</sub>
for no swing and something like:
> τ<sub>mgb1</sub> = 2/3 τ<sub>mgb</sub>
>
> τ<sub>mgb2</sub> = 1/3 τ<sub>mgb</sub>
for swing. Of course, swinging doesn't have to be 2/3, but we can easily model other fractions in similar manner.
https://thoughtstreams.io/jtauber/note-quantization/#card-4473Just as reminder that we're still jus...2014-04-25T00:53:50Z2014-04-25T00:53:50Z
Just as reminder that we're still just talking about the "grid". Actual notes may fall slightly off the grid but our goal (eventually) is to model the grid such that the deltas between note placement and the grid are minimized.
https://thoughtstreams.io/jtauber/note-quantization/#card-4471Perhaps ThoughtStreams needs MathJax ...2014-04-25T00:06:49Z2014-04-25T00:06:49Z
Perhaps ThoughtStreams needs MathJax support so I can do this properly :-)
https://thoughtstreams.io/jtauber/note-quantization/#card-4470I just had a horrible, thought...
...2014-04-25T00:05:43Z2014-04-25T00:05:43Z
I just had a horrible, thought...
Consider something like:
> t<sub>mgb</sub> = τ<sub>mgb</sub>(b - 1)
We're not properly considering the length of earlier beats in a beat group in determining the offset of later beats. Consider G=1, B=3 (which I've suggested above would be 3/4 time).
> t<sub>m11</sub> = 0
>
> t<sub>m12</sub> = τ<sub>m11</sub>
>
> t<sub>m13</sub> = τ<sub>m11</sub> + τ<sub>m12</sub>
So obviously, if τ<sub>m1*</sub> are constant then,
> t<sub>m1b</sub> = τ<sub>m1*</sub>(b - 1)
as before but if not constant we really need to take the sum.
https://thoughtstreams.io/jtauber/note-quantization/#card-4467Or alternatively:
> T<sub>mgbs</su...2014-04-24T15:21:04Z2014-04-24T15:20:54Z
Or alternatively:
> T<sub>mgbs</sub> = T<sub>m</sub> + τ<sub>mg</sub>(g - 1) + τ<sub>mgb</sub>(b - 1) + τ<sub>mgbs</sub>(s - 1)
https://thoughtstreams.io/jtauber/note-quantization/#card-4466But we can now also add:
> T<sub>m...2014-04-24T15:18:48Z2014-04-24T15:17:21Z
But we can now also add:
> T<sub>mg</sub> = T<sub>m</sub> + t<sub>mg</sub> = T<sub>m</sub> + τ<sub>mg</sub>(g - 1)
>
> T<sub>mgb</sub> = T<sub>mg</sub>+ t<sub>mgb</sub> = T<sub>mg</sub>+ τ<sub>mgb</sub>(b - 1)
>
> T<sub>mgbs</sub> = T<sub>mgb</sub> + t<sub>mgbs</sub> = T<sub>mgb</sub> + τ<sub>mgbs</sub>(s - 1)
https://thoughtstreams.io/jtauber/note-quantization/#card-4465Our general equations:
> T<sub>m</...2014-04-24T15:15:00Z2014-04-24T15:15:00Z
Our general equations:
> T<sub>m</sub> = τ<sub>m</sub>(m - 1)
>
> T<sub>mg</sub> = τ<sub>mg</sub>(g - 1)
>
> T<sub>mgb</sub> = τ<sub>mgb</sub>(b - 1)
>
> T<sub>mgbs</sub> = τ<sub>mgbs</sub>(s - 1)
become
> t<sub>m</sub> = τ<sub>m</sub>(m - 1)
>
> t<sub>mg</sub> = τ<sub>mg</sub>(g - 1)
>
> t<sub>mgb</sub> = τ<sub>mgb</sub>(b - 1)
>
> t<sub>mgbs</sub> = τ<sub>mgbs</sub>(s - 1)
https://thoughtstreams.io/jtauber/note-quantization/#card-4464> Let's, denote the number of beat gr...2014-04-24T15:08:33Z2014-04-24T15:08:33Z
> Let's, denote the number of beat groups in measure m by G<sub>m</sub>, the number of beats in beat group g of measure m by B<sub>mg</sub> and the number of sub-beats in beat b or beat group g of measure m by S<sub>mgb</sub>.
>
> If the number of beat groups in a measure is the same regardless of measure, we'll write either G<sub>\*</sub> or just G. Similarly, we can say things like B<sub>\*g</sub> if B varies by beat group but not measure.
>
> We'll similarly use this \* notation with T if possible.
All still true but we'll also use the \* notation with t as well.
https://thoughtstreams.io/jtauber/note-quantization/#card-4463> The b-th beat of the g-th beat grou...2014-04-24T15:03:59Z2014-04-24T15:03:59Z
> The b-th beat of the g-th beat group of the m-th measure, would unsurprisingly be T<sub>mgb</sub> in this model.
It's ambiguous if I'm talking about absolute or relative here but it's T<sub>mgb</sub> or t<sub>mgb</sub> respectively.
> We'll call the level below the beat, the **sub-beat** and it's offset from the beat will be T<sub>mgbs</sub> where s is the sub-beat number within the beat.
The **sub-beat**'s offset from the beat will be t<sub>mgbs</sub> where s is the sub-beat number within the beat.
https://thoughtstreams.io/jtauber/note-quantization/#card-4462> Let's then say that the time-offset...2014-04-24T14:57:02Z2014-04-24T14:57:02Z
> Let's then say that the time-offset of the m-th measure is T<sub>m</sub>.
This remains true.
> We'll say that the time-offset of the g-th beat group of the m-th measure *from the start of the measure* is T<sub>mg</sub>.
>
> Hence the the (sic) absolute time offset of the g-th beat group of the m-th measure would be T<sub>m</sub> + T<sub>mg</sub>.
We'll now say that the time-offset of the g-th beat group of the m-th measure *from the start of the measure* is t<sub>mg</sub>.
Hence the absolute time offset of the g-th beat group of the m-th measure would be T<sub>mg</sub> = T<sub>m</sub> + t<sub>mg</sub>.
https://thoughtstreams.io/jtauber/note-quantization/#card-4460In fact, let's put a stake in the gro...2014-04-24T14:45:09Z2014-04-24T14:45:09Z
In fact, let's put a stake in the ground a decide from this point that `t` means relative time-offset and `T` means absolute time-offset.
This, of course, makes many of the equations earlier now incorrect (or inconsistent with this new notation).
I'll go through and restate some of the major ideas with the new notation (rather than edit earlier thoughts and lose the progression of ideas).
https://thoughtstreams.io/jtauber/note-quantization/#card-4319The previous equations set up a unifo...2014-04-18T10:36:35Z2014-04-17T21:19:17Z
The previous equations set up a uniform grid, but there's no reason not to make τ dependent on m, mg, mgb, and mgbs.
So we end up with something like:
> T<sub>m</sub> = τ<sub>m</sub>(m - 1)
> T<sub>mg</sub> = τ<sub>mg</sub>(g - 1)
> T<sub>mgb</sub> = τ<sub>mgb</sub>(b - 1)
> T<sub>mgbs</sub> = τ<sub>mgbs</sub>(s - 1)
https://thoughtstreams.io/jtauber/note-quantization/#card-4318Let's go back to our simple 120 bpm q...2014-04-18T10:36:18Z2014-04-17T21:01:01Z
Let's go back to our simple 120 bpm quarter notes in 4/4.
We have:
> G = 2
> B = 2
> τ = 2.0 (120 bpm 4/4 = 2 seconds per measure)
and:
> T<sub>m</sub> = τ(m - 1)
> T<sub>\*g</sub> = (τ / G)(g - 1)
> T<sub>\*\*b</sub> = (τ / (GB))(b - 1)
https://thoughtstreams.io/jtauber/note-quantization/#card-4345If we *do* use a different notation (...2014-04-18T09:32:19Z2014-04-18T09:32:19Z
If we *do* use a different notation (perhaps t vs T) for absolute time-offset versus time-offset from the most recent tick of the grid-level above, then note that, in the above, the measure-level would be notated differently to the lower-levels.
If we introduced grid-levels above the measure (phrase, theme, theme group, section, movement, etc) then the measure-level would become relative.
https://thoughtstreams.io/jtauber/note-quantization/#card-4320Note that we don't need to divide by ...2014-04-17T21:21:02Z2014-04-17T21:21:02Z
Note that we don't need to divide by G or B as before because that's baked into τ<sub>mg</sub> and τ<sub>mgb</sub> respectively.
In our constant tempo version,
> τ<sub>m</sub> = τ
> τ<sub>mg</sub> = τ / G
and so on.
https://thoughtstreams.io/jtauber/note-quantization/#card-4317Let's, denote the number of beat grou...2014-04-17T21:02:59Z2014-04-17T20:54:31Z
Let's, denote the number of beat groups in measure m by G<sub>m</sub>, the number of beats in beat group g of measure m by B<sub>mg</sub> and the number of sub-beats in beat b or beat group g of measure m by S<sub>mgb</sub>.
If the number of beat groups in a measure is the same regardless of measure, we'll write either G<sub>\*</sub> or just G. Similarly, we can say things like B<sub>\*g</sub> if B varies by beat group but not measure.
We'll similarly use this \* notation with T if possible.
https://thoughtstreams.io/jtauber/note-quantization/#card-4314The b-th beat of the g-th beat group ...2014-04-17T21:02:34Z2014-04-17T20:34:18Z
The b-th beat of the g-th beat group of the m-th measure, would unsurprisingly be T<sub>mgb</sub> in this model.
We'll call the level below the beat, the **sub-beat** and it's offset from the beat will be T<sub>mgbs</sub> where s is the sub-beat number within the beat.
https://thoughtstreams.io/jtauber/note-quantization/#card-4312Let's then introduce a grid level dir...2014-04-17T21:02:19Z2014-04-17T20:30:38Z
Let's then introduce a grid level directly below the measure but above the beat. I'll call this the **beat group**. The idea here is that the 4 beats of a 4/4 measure can be thought of as two groups of two. Similarly, something like 5/8 can be thought of as a two-beat group followed by a three-beat group or vice versa.
We'll say that the time-offset of the g-th beat group of the m-th measure *from the start of the measure* is T<sub>mg</sub>.
Hence the the absolute time offset of the g-th beat group of the m-th measure would be T<sub>m</sub> + T<sub>mg</sub>.
https://thoughtstreams.io/jtauber/note-quantization/#card-4283Say a piece begins at one tempo, τ<su...2014-04-17T21:01:59Z2014-04-13T05:48:04Z
Say a piece begins at one tempo, τ<sub>1</sub> and then changes to τ<sub>2</sub> instantaneously.
We'll model this as two sections, each with it's own equation:
> t<sub>1i</sub> = b<sub>1i</sub>τ<sub>1</sub> + T<sub>1</sub>
> t<sub>2j</sub> = b<sub>2j</sub>τ<sub>2</sub> + T<sub>2</sub>
Here t<sub>1i</sub> means the time-offset of the i-th note in section one. b<sub>1i</sub> maps notes in section one to beat numbers. T<sub>1</sub> tells us the time offset of the start of the first full bar in the first section.
And the same for the second section, replacing 1 with 2. In the above, I've also used j instead of i to make more explicit that it ranges over a different set of numbers (although I will not always do that).
Note that T<sub>2</sub> is basically the total length of the first section plus the pause between the sections if any.
https://thoughtstreams.io/jtauber/note-quantization/#card-4316One hypothesis is that from measure o...2014-04-17T20:44:58Z2014-04-17T20:39:52Z
One hypothesis is that from measure on down, each hierarchy either splits into 2 or 3. Open questions are how tuplets are to be modeled and also whether something like 13/8 would need multiple levels of beat group.
But I don't think we're relying on that hypothesis here anyway.
https://thoughtstreams.io/jtauber/note-quantization/#card-4315It is worth noting that the differenc...2014-04-17T20:37:08Z2014-04-17T20:37:08Z
It is worth noting that the difference between 3/4 and 6/8 in this model is that a 3/4 measure consists of 3 beats each made up of 2 sub-beats and a 6/8 measure consists of 2 beats each made up of 3 sub-beats.
Hence simple vs compound time is distinguished by 2 or 3 sub-beats per beat.
Note that the notion of a beat group is degenerate in this case and is only useful in cases were the number of beats per measure is more than three.
https://thoughtstreams.io/jtauber/note-quantization/#card-4313I'm undecided when to use t vs T at t...2014-04-17T20:31:55Z2014-04-17T20:31:55Z
I'm undecided when to use t vs T at the moment (perhaps one should be absolute and the other relative to start of the previous level of the hierarchy; we'll come back to all this)
https://thoughtstreams.io/jtauber/note-quantization/#card-4311Although we'll sometimes have grid le...2014-04-17T20:29:04Z2014-04-17T20:23:48Z
Although we'll sometimes have grid levels above the measure (as we saw earlier with sections at different tempi), let's imagine for now that the top level is the measure level.
Let's then say that the time-offset of the m-th measure is T<sub>m</sub>.
https://thoughtstreams.io/jtauber/note-quantization/#card-4282Now let's remove the assumption that ...2014-04-17T20:22:12Z2014-04-13T05:43:20Z
Now let's remove the assumption that the tempo is the same throughout the piece. We'll start with handling sections of different tempi, then discuss ritardando. We'll delay discussion of rubato for the moment.
https://thoughtstreams.io/jtauber/note-quantization/#card-4310So let's develop our model further to...2014-04-17T20:18:00Z2014-04-17T20:18:00Z
So let's develop our model further to support hierarchy.
We'll initially focus purely on the time-offset of various points on a multi-level *grid* before adding the mapping of notes to that grid.
https://thoughtstreams.io/jtauber/note-quantization/#card-4309If we model the tempo of different se...2014-04-17T20:12:36Z2014-04-17T20:11:31Z
If we model the tempo of different sections in this way, why not then model each measure this way?
This would allow for all sorts of variation within a measure without affecting the tempo at the measure-level grid.
The same applies to notes within the beat-level of the grid.
https://thoughtstreams.io/jtauber/note-quantization/#card-4280We can quickly accommodate pick ups a...2014-04-13T05:48:49Z2014-04-13T05:37:58Z
We can quickly accommodate pick ups and a silence before the first event as follows:
* let T be the time offset of the start of the first full measure
* allow negative b<sub>i</sub> for pick ups / anacrusis
Only the first affects our equation, which becomes:
> t<sub>i</sub> = b<sub>i</sub>τ + T
To given an example, if there's a one beat pickup, b<sub>1</sub>would equal -1.
https://thoughtstreams.io/jtauber/note-quantization/#card-4281To be clear: we're not doing quantiza...2014-04-13T05:44:31Z2014-04-13T05:44:31Z
To be clear: we're not doing quantization yet, we're just building a model. Once we have a model, it will be a lot easier to discuss how the parameters of that model might be inferred from a performance.
https://thoughtstreams.io/jtauber/note-quantization/#card-4264So our archetypal relationship betwee...2014-04-13T05:32:59Z2014-04-11T13:18:22Z
So our archetypal relationship between the "beat grid" and time offsets is:
> t = nτ
where t is the time offset, n is the beat number and τ is the tempo.
https://thoughtstreams.io/jtauber/note-quantization/#card-4279Now, of course, we really want to rel...2014-04-13T05:32:46Z2014-04-13T05:32:31Z
Now, of course, we really want to relate the time offset with an event number so we need a mapping of event number to beat number. Let's use b<sub>i</sub> to denote the beat number of the i-th note.
We then have
> t<sub>i</sub> = b<sub>i</sub>τ
https://thoughtstreams.io/jtauber/note-quantization/#card-4262Given the case outlined above, the pe...2014-04-11T09:54:00Z2014-04-11T09:54:00Z
Given the case outlined above, the performance might look something like this (remember we're only considering the time offset of each event):
0s, 0.5s, 1.0s, 1.5s, 2.0s, ...
Given the constraints we initially specified above, this can only be a series of quarter-notes at 120 bpm.
https://thoughtstreams.io/jtauber/note-quantization/#card-4261There are two other assumptions we're...2014-04-11T09:49:03Z2014-04-11T09:47:13Z
There are two other assumptions we're going to make initially.
Firstly, we're going to assume common time: four simple beats to a measure.
Secondly, we're going to assume that our tempo lies between 70 bpm and 140 bpm. In other words, a performance at 150 bpm will be interpreted at 75 bpm with note lengths half of what they would be viewed as under 150 bpm.
https://thoughtstreams.io/jtauber/note-quantization/#card-4258I'm deliberately avoiding *duration* ...2014-04-11T09:43:28Z2014-04-11T09:23:13Z
I'm deliberately avoiding *duration* initially because I want to pursue placing notes on the grid (and, indeed *inferring* the grid to start with) before any discussion about note duration.
Note duration is hugely important to a lot of applications (not least of which the kind of analysis of articulation I want in Czerny) but I think we can proceed a long way before considering them.
It's also possible that velocity will have role to play in identify the time signature but again, we can defer that possibility for a while.
https://thoughtstreams.io/jtauber/note-quantization/#card-4255In the case of durations and rhythms ...2014-04-11T09:35:47Z2014-04-11T08:57:12Z
In the case of durations and rhythms (which is our focus in this stream) we go from offsets (say in seconds) to measures and beats.
https://thoughtstreams.io/jtauber/note-quantization/#card-4259Let's started with the simplest possi...2014-04-11T09:30:17Z2014-04-11T09:27:15Z
Let's started with the simplest possible case: a series of events with uniform rhythm, aligned perfectly with the grid, with uniform tempo, no anacrusis / pick up, and with the time offset of the first note equal to zero.
This may seem ridiculously simple (and almost useless) but it will allow us to define some terms and set things up.
We can then successively remove each of these simplifications.
https://thoughtstreams.io/jtauber/note-quantization/#card-4257First, let's define what I mean by a ...2014-04-11T09:19:26Z2014-04-11T09:19:26Z
First, let's define what I mean by a "performance".
By **performance** I mean a set of events that include at least a time offset.
Typically events will also include pitch information and possibly other things such as velocity (in the case of a MIDI performance) but none of these will enter into our discussion.
https://thoughtstreams.io/jtauber/note-quantization/#card-4256Even though the main point of quantiz...2014-04-11T09:06:04Z2014-04-11T09:06:04Z
Even though the main point of quantization is dealing with notes that *aren't* "exactly on the grid" there are still some preliminary issues we need to deal with even in the case where a performance *is* exactly aligned with the grid.
https://thoughtstreams.io/jtauber/note-quantization/#card-4254In the case of pitch, we go from freq...2014-04-11T08:54:29Z2014-04-11T08:54:29Z
In the case of pitch, we go from frequency to letter name + octave via a choice of tuning and temperament, then abstract away the octave and factor out the key to get an abstraction like "the 3rd note of the scale" or a "IV<sup>6</sup><sub>4</sub> chord" or whatever.
https://thoughtstreams.io/jtauber/note-quantization/#card-4253One of the fundamental aspects of mus...2014-04-11T08:48:42Z2014-04-11T08:48:42Z
One of the fundamental aspects of music theory is that we deal not in frequencies and clock timings but more abstractly in pitches (or scale degrees) and rhythms set against a grid.
https://thoughtstreams.io/jtauber/note-quantization/#card-4252I should note that, while quantizatio...2014-04-11T08:44:53Z2014-04-11T08:44:53Z
I should note that, while quantization is often associated with *fixing* mistakes in performances, that is neither my interest, nor I suspect Adrian's.
Rather I'm interested in taking a representation of a performance and non-destructively calculating a quantized version to both answer questions about this quantized version (e.g. tempo and tempo changes) and also analyze style (in much the same way as Czerny is intended to help with, albeit in the presence of a score in the Czerny case).
https://thoughtstreams.io/jtauber/note-quantization/#card-4251In Czerny, I largely side-stepped the...2014-04-11T08:42:11Z2014-04-11T08:42:11Z
In Czerny, I largely side-stepped the issue of quantization between the alignment of notes that I do doesn't even look at note start times, only note order (at least for now).
Plus with Czerny, I assume there's a representation of the score, whereas the problem of note quantization generally assumes there's no reference score (and I'll make that assumption in what follows).
https://thoughtstreams.io/jtauber/note-quantization/#card-4250Now I'm sure there's academic literat...2014-04-11T08:40:18Z2014-04-11T08:40:18Z
Now I'm sure there's academic literature on this but before I dive into that, I wanted to give some it in-depth thought of my own. This thought stream is my initial place for notes (no pun intended).
https://thoughtstreams.io/jtauber/note-quantization/#card-4249Last week Adrian Holovaty asked me (i...2014-04-11T08:39:10Z2014-04-11T08:39:10Z
Last week Adrian Holovaty asked me (in response to a comment about me still wanting to write a guide to music theory for programmers—not sure if Adrian knows about Czerny) about algorithms for note quantization.
As it's of interest to me and somewhat related to Czerny, I decided I'd put down some thoughts.
https://thoughtstreams.io/jtauber/note-quantization/#card-4248At PyCon last year I talked about my ...2014-04-11T08:36:23Z2014-04-11T08:36:23Z
At PyCon last year I talked about my project [Czerny](https://github.com/jtauber/czerny) which I announced four years ago but haven't really worked on much since.
The idea of Czerny was to align representations of performances with representations of score (particularly with Piano music) to both (a) assess errors; (b) study articulation, timing variations, etc.