This is an attempt at creating a tool that could reduce the sampling rate of an audio signal without introducing common artifacts such as constant aliases or low-pass reconstruction filtering (but add other funnier artifacts instead).
Here is the source code on GitHub.
I only compiled the Windows version but since it's Java/Processing it could probably work on other systems as well, it will probably only need Minim (used for audio playback) as an additional library.
How does it work?
By default, it acts like a normal sample-rate-reducer with an arbitrary frequency.
However, this frequency will dynamically adapt to the input sample according to something I'll refer to as an "optimization" method.
Typically, the frequency will increase as the sample gets more detailled and decrease as the sample gets more simple (although that's not entirely true for all cases).
Then audio gets reconstructed using one of the available interpolation functions (including a few funky ones).
Since the sampling frequency can vary continuously, the artifacts tend to be more "noisy" and less "hissy" compared to traditional decimators.
While the effects of constant resampling can be quite simply understood in terms of frequency space, I suppose variable sampling complicates the frequency-domain transformation.
Checking the precision of the reconstructed signal and modifying parameters accordingly may sound like an obvious process though, so I'm guessing something similar probably already exists or has been studied in some way and I don't know about it yet.
The closest effect I could think of would probably be good old dfx geometer.
The "sinus" option sounds especially reminiscent of it, I am also using the term "landmark" in the code instead of the ambiguous word "sample".
How does it sound?
Here is a test sample :
And here it is processed with various subtle to extreme parameters :
What does the sound look like?
Video demo:
More explanations about the parameters:
Frequency
If you plan to use the other specific features, you should probably keep this one as low as possible so it's mainly there as a guardrail.
Optimization
There are currently four optimization modes:
- off : only the fixed frequency is used
- integral : adds sample points to ensure the difference between the input and output remains below a certain threshold
- gap : adds sample points to ensure the difference between one sample point and the next remains below a certain threshold
- zero : adds sample points at zero-crossings, can also skip some or subdivide durations using the threshold slider
Some of these methods use a brute-force approach where they will try every possible sampling length until their condition for stopping is fulfilled.
If the methods fails to find a satifying point, it will always default to the fixed frequency slider as a maximum number of sample.
Compression
The dynamic range of the input sample can be temporarily reduced while computing the sampling points and then uncompressed during reconstruction.
This has been set up to compensate the fact that quiet parts of the audio tend to be less optimized and can disappear under the crushing.
Turning this slider very close to zero will actually expand the audio instead of compressing it.
Interpolation
Interpolation methods (from simple to weird):
- Sample and hold (the usual staircase)
- Linear (flat ramp)
- S-shaped curve (slightly smoother than linear)
- Sawtooth (somehow like linear but always from low to high)
- Velvet (only a single sample used at the input value, then zeroes until the next sample)
- Zero (always zero)
Sine
This option adds an arbitrary single full sine cycle between each sample point, scaled by the points gain difference.
IIR
Applies a low-pass filter.
I earlier said that a reconstruction filter was one of the things I wanted to avoid but there is a small twist here : the sine and IIR passes are applied during the optimization process, which means that if, for instance, the use of a sine or a filter drives the output further away from the input at some point in "integral" mode, the sampling frequency may temporarily increase there to compensate.
What else can we use it for?
When selecting "no" optimization, it can be used as a normal bitcrusher.
Then some of the uncommon interpolation modes (such as "sawtooth" or "velvet noise") can make it sound like a modulated or hard-synced synth.
Example with an original signal followed by several processed alternatives:
Many extreme sets of parameters can sound like organic distortion.
Example with an original signal followed by several processed alternatives:
If you set the frequency very low, both optimization and interpolation to "zero", and move the "sine" slider away from zero, the sample will be recreated purely from sines.
In that case, the "threshold" slider will transpose the pitch. Very high pitches tend to produce "wet" sounding noises.
Example with an original signal followed by several processed alternatives:
Why is it not real-time?
I deliberately chose not to make this tool real-time. Since some developers seem to think real-time is always necessarily better, let me point out a few reasons why I made this choice:
- It was probably quicker for me to write and not care about buffers and latency.
- Because of the use of a threshold that gets reached over time, the "look ahead" time of the optimisation process can vary a lot. I could even extend it to as long as I wanted if necessary since the whole audio signal is known in advance.
- Very complex cases will take all the time they need to brute-force all possibilities before processing the sound, sometimes a lot longer than its actual duration, but it's not going to use more cpu or introduce crackles and lagging.
- You can load a large batch of files inside and process them all in parallel with the same settings (or different settings).
- Seeing the whole waveform graphically change at once while tweaking the parameters can be very informative and enjoyable.
Why does everything look like it has been made decades ago ?
Win-95 GUI is still one of my favorite software aesthetics but I wrote the code in 2023.
Known limitations:
I primarly made this tool for myself and then later decided to make it somehow more user-friendly and release it, but its workflow is still very specific and some use cases aren't nicely handled.
Here is, as I'm writing this, a list of known improvable things (mainly as a reminder to myself).
- very long files can require too much memory and crash the tool
- only 16 bits .wav file will load, it's supposed to handle 8 and 24 but apparently doesn't
- for practical reasons, I have allowed multiple threads to process multiple sounds simultaneously but it's not robust and, if you mess with all the buttons at once, weird (but not cool-weird) things may happen
- the use of sliders while listening to a processed sample will not have any impact on what's currently playing, it will need to be reprocessed first, this is by design but the GUI doesn't make it obvious enough
- I'm performing most of the signal computations using doubles, but this is invalidated by a few math functions that don't accept doubles as arguments
- the user should be allowed to input precise numbers with a keyboard, I hate it when VSTs don't let me do that so I shouldn't overlook it in my own tool
- the way intermediate zero-crossing points and fixed-frequency points overlap each-other is surely not optimal
- the leftmost part of the display often looks wrong, especially on zero-crossing mode, the rightmost part also does on some other modes
- it would be cool to be able to zoom/unzoom on the display but that's a risky thing since I'm only processing the visible part of it
- I believe the "gap" behavior is correct, but it might seem counterintuitive as it always favors a new fitting point closest to the previous sample point, not the next
- In "zero" mode, the sampled values are those right before and right after zeroes, so they're not exactly zero. The added sines are only audible because they're scaled by these tiny values.
Further development:
Since I've made this tool, I've used it in my work a few times already. I may want to include other features in the future.
- smooth frequency movement : the frequency values themselves could be filtered instead of jumping directly from one to the next, I believe this could help balance the noise bring back some tonal but moving artifacts.
- even though it was not part of my original plan, I'm really happy about the "sine" option and I could think of other more sophisticated way to generate generate signals such as reusing bounded waveforms from previous parts of the input.
- with a bit of debugging and cleaning I could remove all the specific logic from the tool and just keep the interface as a template for future audio experiments.
Bonus:
I have created a batch of drum samples and pads by running very simple sounds through SlideCrusher.
Here they are along with two Ableton devices that use them.
