Filtering Latency - Encountering a Guardian of the Threshold.
A Guardian of the Threshold is a
menancing figure, which manifests itself as soon as
the student of the spirit ascends upon the path to higher worlds of Knowledge.
In our journey of DSP there will be many of these, today we deal with Filtering Latency.
You see previusly in our code we tried to filter the samples as we received them, but alas you can’t do this, it takes time to filter and while you are filtering the audio still needs to be sampled!
Here is how we are doing it
This however ends up crashing our app!
[00:05:05.874,938] <wrn> ble_module: Connection estalished!
[00:05:05.874,969] <wrn> fsm_module: INCOMING EVENT 0x00000001
[00:05:05.875,274] <wrn> fsm_module: Go to connected State
[00:05:05.875,274] <wrn> fsm_module: Left Idle
[00:05:05.875,274] <wrn> fsm_module: Entering Connected state
[00:05:05.875,366] <wrn> audio_module: SENSING AUDIO
[00:05:05.979,370] <wrn> audio_module: Consumed Samples
[00:05:06.173,034] <err> i2s_nrfx: Failed to allocate next RX buffer: -12
[00:05:06.364,624] <wrn> audio_module: Filtered a Block
[00:05:06.369,384] <wrn> audio_module: Consumed Samples
[00:05:06.756,347] <wrn> audio_module: Filtered a Block
[00:05:06.761,108] <wrn> audio_module: Consumed Samples
[00:05:07.145,751] <wrn> audio_module: Filtered a Block
[00:05:09.145,904] <wrn> audio_module: SENSING AUDIO
[00:05:09.249,908] <wrn> audio_module: Consumed Samples
[00:05:09.638,824] <wrn> audio_module: Filtered a Block
[00:05:09.643,798] <wrn> audio_module: Consumed Samples
[00:05:09.741,271] <err> i2s_nrfx: No room in RX queue
[00:05:09.741,271] <err> i2s_nrfx: No room in RX queue
[00:05:10.031,982] <wrn> audio_module: Filtered a Block
[00:05:10.037,017] <wrn> audio_module: Consumed Samples
[00:05:10.424,774] <wrn> audio_module: Filtered a Block
As you can see the RX Queue ends up running out of memory, and this leads to a crash. Intuition says the RX queue is filling up because of the delay between filtering and receiving
- Can we pass filtering to another thread and handle it there?
- Do we need to mess with priorities between filtering and sensing?
can we use the state machine, to transition from capturing audio to filtering audio.
Plan
- Add a filtering Thread, with its own MEM_SLAB
- copy block samples to filter blocks
- Apply the filter in our new thread
- Use hysteresis detector in filter thread to maintain noise/ no noise state
- Do inferece in filter thread.
Questions
- How can we ensure that we do Inference right after recording? FSM perhaps?
- What sort of deadline can we adhere to with the Inference?
- Is there anyway of making this faster?