AudioMixSettings interface

The settings for an AudioMix operation see: NorskTransform.audioMix()

Signature:

export interface AudioMixSettings<Pins extends string> extends ProcessorNodeSettings<AudioMixNode<Pins>>

Extends: ProcessorNodeSettings<AudioMixNode<Pins>>

Properties

Property Type Description

Property	Type	Description
channelLayout	ChannelLayout	The channel layout that the mixer runs at all audio streams will be normalised to this value and therefore this will be the output channel layout of this node
maxQueueMs?	number	(Optional) Opt-in: the longest the mix waits for a pin that is behind. By default the mix waits for every pin (for up to 10 s), so it runs as late as its latest pin: one talker 3 s behind delays everything this mixer puts out by 3 s, and a pin that stops sending holds it up altogether. With this set, the mixer paces itself by a clock of its own, which runs in real time (on the engine’s clock, which the inputs' timestamps are on) and cannot stall. A pin more than `maxQueueMs` behind that clock no longer holds the mix back: the mix carries on without it, as if it were silent, and the audio it sends for moments that have already been mixed is discarded, not mixed late. When it catches up to within `maxQueueMs` it is back in the mix. A pin that sends nothing for longer than `maxQueueMs` is treated the same way. A pin whose timestamps run AHEAD of real time (after a jump forward, say) does not make the others late: it waits for the clock, and is mixed when its timestamps come round. Its audio more than 10 s ahead is discarded. What it costs: whenever a pin is late the mix runs up to `maxQueueMs` (plus about a frame, 40 ms) behind real time, so keep it small for low latency; and a pin that is always more than `maxQueueMs` behind is never heard, so make it larger than the lag you are prepared to wait for (a source that delivers in bursts loses each burst that arrives late). `maxQueueMs` counts from when the audio was received by Norsk, so it has to cover the time the audio takes to get to the mixer as well (decoding, and any jitter buffer on the way): set lower than that, every pin is late and the mix is silent.
onInboundContextChange?	(allStreams: Map<Pins, StreamMetadata[]>) ⇒ Promise<void>	(Optional) Called when the set of streams being mixed changes: a pin’s audio has arrived and is about to join the mix, a pin has left it, or a pin’s stream has changed. The mixer is stopped while this runs. Nothing from the new set of streams has been mixed yet, and nothing will be until the returned promise settles - so keep it short. An AudioMixNode.updateConfig() made before it settles is applied on exactly the first frame of the new set: a pin muted here as it joins is never heard, and a pin dropped from `sources` here as it leaves is dropped on the frame it leaves on. (If several are made, the last one is used.) Audio that arrived under the previous set, including the last of a pin that has been removed, is mixed with the previous gains. The mixer stops only when this callback is given. Without one, a change in the set of streams goes straight through, and an `updateConfig` is applied when it arrives. This is exact within this node. Another node given the same pin at the same moment (a second mixer, a StreamSwitchSmoothNode) stops at its own first frame of that pin, which will be close but is not guaranteed to be the same frame. Lining a change up across several nodes - a "take" that cuts the picture in a StreamSwitchSmoothNode and brings the same source up in one or more mixers: - Best: subscribe the source to every node ahead of time - not the active source in the switch, muted in the mixers - and make the take with plain calls issued together (`switchSource`, `updateConfig`). With nothing joining at that moment no node has to stop, and the changes land within about a frame of each other. - If the source can only be subscribed at the moment of the take: subscribe it to all the nodes in the same tick, and have each node make its own change in its own `onInboundContextChange`. Each then lands on that node’s first frame of the source; the nodes agree to within a frame or two. - Do not make one node’s change from another node’s callback. The other node is not stopped for it, and the change lands wherever that node has got to. - Do not make the callbacks wait for each other. Where each node stops is settled when the subscription reaches it, not when its callback returns, so waiting brings them no closer; it only keeps every node stopped for as long as the slowest, and for ever if one of them is never called (a source with no audio, say). - A node stops its whole output while a source that is behind the others joins it, whichever way the take is made, for as long as that source is behind. If that pause matters, hold the sources that are always present back by more than the live ones lag (a NorskTransform.jitterBuffer() on them, at the cost of that much delay), so that a joining source is never the one being waited for. Or bound the wait in the mixer with AudioMixSettings.maxQueueMs, at the cost of not hearing a source while it is further behind than that. - Where picture and sound have to change on the very same frame, take the sound from the switch itself, which keeps the two together; use separate mixers for feeds where a frame or two is acceptable.
outputSource	string	The source name to use for the output stream
sampleRate?	SampleRate	(Optional) The sample rate that the mixer runs at all audio streams will be normalised to this value and therefore this will be the output sample rate of this node
sources	readonly AudioMixSource<Pins>[]	The audio sources to mix. If this lists any pins, list every pin that should be heard: a pin that is subscribed but not listed is silent (and the SDK and the engine each log a warning if it stays that way for a second). A pin listed without `channelGains` is mixed as it comes. With an empty list, every subscribed pin is mixed.

channelLayout

ChannelLayout

The channel layout that the mixer runs at all audio streams will be normalised to this value and therefore this will be the output channel layout of this node

maxQueueMs?

number

(Optional) Opt-in: the longest the mix waits for a pin that is behind.

By default the mix waits for every pin (for up to 10 s), so it runs as late as its latest pin: one talker 3 s behind delays everything this mixer puts out by 3 s, and a pin that stops sending holds it up altogether.

With this set, the mixer paces itself by a clock of its own, which runs in real time (on the engine’s clock, which the inputs' timestamps are on) and cannot stall. A pin more than maxQueueMs behind that clock no longer holds the mix back: the mix carries on without it, as if it were silent, and the audio it sends for moments that have already been mixed is discarded, not mixed late. When it catches up to within maxQueueMs it is back in the mix. A pin that sends nothing for longer than maxQueueMs is treated the same way.

A pin whose timestamps run AHEAD of real time (after a jump forward, say) does not make the others late: it waits for the clock, and is mixed when its timestamps come round. Its audio more than 10 s ahead is discarded.

What it costs: whenever a pin is late the mix runs up to maxQueueMs (plus about a frame, 40 ms) behind real time, so keep it small for low latency; and a pin that is always more than maxQueueMs behind is never heard, so make it larger than the lag you are prepared to wait for (a source that delivers in bursts loses each burst that arrives late). maxQueueMs counts from when the audio was received by Norsk, so it has to cover the time the audio takes to get to the mixer as well (decoding, and any jitter buffer on the way): set lower than that, every pin is late and the mix is silent.

onInboundContextChange?

(allStreams: Map<Pins, StreamMetadata[]>) ⇒ Promise<void>

(Optional) Called when the set of streams being mixed changes: a pin’s audio has arrived and is about to join the mix, a pin has left it, or a pin’s stream has changed.

The mixer is stopped while this runs. Nothing from the new set of streams has been mixed yet, and nothing will be until the returned promise settles - so keep it short. An AudioMixNode.updateConfig() made before it settles is applied on exactly the first frame of the new set: a pin muted here as it joins is never heard, and a pin dropped from sources here as it leaves is dropped on the frame it leaves on. (If several are made, the last one is used.) Audio that arrived under the previous set, including the last of a pin that has been removed, is mixed with the previous gains.

The mixer stops only when this callback is given. Without one, a change in the set of streams goes straight through, and an updateConfig is applied when it arrives.

This is exact within this node. Another node given the same pin at the same moment (a second mixer, a StreamSwitchSmoothNode) stops at its own first frame of that pin, which will be close but is not guaranteed to be the same frame.

Lining a change up across several nodes - a "take" that cuts the picture in a StreamSwitchSmoothNode and brings the same source up in one or more mixers:

- Best: subscribe the source to every node ahead of time - not the active source in the switch, muted in the mixers - and make the take with plain calls issued together (switchSource, updateConfig). With nothing joining at that moment no node has to stop, and the changes land within about a frame of each other. - If the source can only be subscribed at the moment of the take: subscribe it to all the nodes in the same tick, and have each node make its own change in its own onInboundContextChange. Each then lands on that node’s first frame of the source; the nodes agree to within a frame or two. - Do not make one node’s change from another node’s callback. The other node is not stopped for it, and the change lands wherever that node has got to. - Do not make the callbacks wait for each other. Where each node stops is settled when the subscription reaches it, not when its callback returns, so waiting brings them no closer; it only keeps every node stopped for as long as the slowest, and for ever if one of them is never called (a source with no audio, say). - A node stops its whole output while a source that is behind the others joins it, whichever way the take is made, for as long as that source is behind. If that pause matters, hold the sources that are always present back by more than the live ones lag (a NorskTransform.jitterBuffer() on them, at the cost of that much delay), so that a joining source is never the one being waited for. Or bound the wait in the mixer with AudioMixSettings.maxQueueMs, at the cost of not hearing a source while it is further behind than that. - Where picture and sound have to change on the very same frame, take the sound from the switch itself, which keeps the two together; use separate mixers for feeds where a frame or two is acceptable.

outputSource

string

The source name to use for the output stream

sampleRate?

SampleRate

(Optional) The sample rate that the mixer runs at all audio streams will be normalised to this value and therefore this will be the output sample rate of this node

sources

readonly AudioMixSource<Pins>[]

The audio sources to mix.

If this lists any pins, list every pin that should be heard: a pin that is subscribed but not listed is silent (and the SDK and the engine each log a warning if it stays that way for a second). A pin listed without channelGains is mixed as it comes. With an empty list, every subscribed pin is mixed.