Separate Auto DJ For Decks 3+4 With Exclusive Auto DJ Crates Playlist For Use In Live Generative DJing?

[GomuSkelly]

Feature Description

Wondering if it would be possible to enable Auto DJ working for Decks 3+4, with the ability to use a separate Auto DJ Crates location + playlist in the Library for Decks 3+4?

We've just released a basic template using ComfyUI and Mixxx to create live generative dj sets, and are hoping it will soon be possible to demo 4 deck Auto DJing while generating both music and sound effects in realtime. The extra generated sampler sfx would be mixed for added texture on Decks 3+4. Currently we don't see a way of setting 3+4 to play from a separate playlist.

[GomuSkelly]

Here's some links for MozAIk:

https://github.com/GomuSkelly/MozAIk

https://www.youtube.com/watch?v=hdIFKt8geU8

https://www.youtube.com/watch?v=nqwre61Yz7M

https://www.reddit.com/r/comfyui/comments/1hrer1e/introducing_mozaik_a_realtime_generative_avj/

We're keeping the template releases simple for now because we want it to be as easy as possible for beginners to get up and running with the technique, but we've been building the broader system out for months so there's more coming with Mixxx at the core. It would be great to have a way of layering in additional generative sonic texture from a separate Auto DJ Crates playlist on Decks 3+4.

Also, thanks for building such an amazing open source tool! Love it :)

[daschuer]

Many things are possuble to add to Mixxx since it is free and open source. However you need a contributor who is able to integrate that.

I have not yet understand you use case. Do you want two independent Auto DJs? What is the responsibility of the first and the second one?

[GomuSkelly]

Yes, if possible, two independent (but 'collaborative') Auto DJ's.

For example - in our specific use case Auto DJ 1 would control Decks 1+2, and mix music as usual, with Sync enabled for beatmatching. On Decks 3+4, Auto DJ 2 would only play the audio (in our case generative samples of sonic texture rather than music for Decks 3+4) that was dragged into Auto DJ 2's separate playlist.

The ability to choose whether to separate Sync function for Auto DJ 2 / Decks 3+4 would also be useful, as the samples in Auto DJ 2's Crates would not have a musical rhythm.

In other scenarios we generate rhythm without melodic elements, and so in that case being able to load up all 4 decks and have them all sync tempo would be of benefit. Thinking of something along the lines of a dual Auto DJ B2B Mode (back to back).

Thanks for taking a look, and let us know if there's anything in the description that still remains unclear.

[daschuer]

Ah, I understand. However the overall use case form user and musical perspective is still unklear. Why would one need that.

Maybe we can find a solution that also helps radio DJs and Tango DJs.

• Tango DJs play cortina tracks between normal dance tracks for changing couples
• Radio DJs like to add jingles or comercials according to a certein rule.

#7932
#13754
#10709
#11727
#6796
#10434

So both have also diffrent types of Tracks where Auto DJs should be aware of 'collaborative'. Are some ideas oultlined in the linked bugs resonate with your ideas?

[GomuSkelly]

I read through the linked issues. There's a lot to synthesize!

Using Mixxx for broadcasts, we load our premade drops/jingles/etc on the 64 samplers and trigger those with MIDI controllers. I can see how having a second Tracks location could help Radio DJ's distinguish between music and jingles/commercials material, and it would be great to have rules for when that material triggers from the 64 sampler slots. We could be hands off with that side of things if needed, which ties in with our use case for realtime generative DJing with Mixxx.

We currently use Auto DJ to play the music being generated in the backgrund in ComfyUI on Decks 1+2. Using CTRL+SHIFT+L, we frequently update the Library. A rule to set a scheduled auto-Library Rescan would be very useful also, as there's new material, flacs of 48 seconds length per file, being generated every 10-15 seconds in our use case.

We manually Analyse the new musical material that appears in Tracks upon Library Rescan, then the material that has generated at the requested BPM (we prompt for 160bpm) is then manually dragged into Auto DJ.

Analyse provides a 80/160 BPM tempo for most material generated with "160bpm" in the text prompt. The material that generates at a random BPM, we just delete/hide from Tracks.

Using Decks 3+4 for playing sample generations (non-musical sfx and sonic texture for layering), we don't add this to Auto DJ's playlist as we always require music generations to be loaded on Decks 1+2.

We dont analyse non-music samples, and presently we drag them on to Decks 3+4 and play them manually.

Presently we distinguish between music and texture by the filename, however having a separate Tracks location for the sonic texture, and a second Auto DJ playlist would allow us to distinguish between/separate the material and ensure Decks 1+2 always have music, and Decks 3+4 always Auto DJ 2's either sonic texture, or can layer in additional rhythmic or melodic elements.

Would it be of benefit if we link you a video of our current 4 Deck process in action? I could record a quick clip in 4k (or 1080 if preferred) and upload it later today. I can show examples of 4 Decks with music/rhythm on each deck, and Deck 1+2 Auto DJing music while the non-music samples are manually added to Decks 3+4.

[daschuer]

Yes, that's helpful for a potential contributor wo likes to implement that.

It is however doubtful that one pops up to implement such a big change. Do you have interest to do it yourself? I think a key for success it to slit the project into small self contained changes.

[GomuSkelly]

Ok will put a clip together later and link it here.

I'm not an experienced programmer but my partner has knowledge in that area.

In your opinion, how feasible/challenging to implement would the main feature suggestion of a second Auto DJ be?

[daschuer]

I don't think that a pure concept of a second Auto DJ feasible. But I think extending the current Auto DJ to use four decks according your rules can be done.
Doing the whole thing at once is challenging and has a high risk to fail. But doing it step by step will work.
You may start with a low hanging fruit.

[GomuSkelly]

Ok thanks, that sounds like a promising approach.

We took a quick look at the languages used in Mixxx. If we're looking for a low hanging fruit to contribute towards some of the required adaptations, would it be C++ for that work?

To my mind there's a few discreet features that would contribute to our intended use case, but be of utility for other users also:

Auto-Library Rescan conditions. As mentioned, we rescan the library with Ctrl+Shift+L regularly. Implementing an option, to define in seconds/minutes/hours the time between Mixxx automatically rescanning the Library would be of benefit.

Auto-Analyse newly scanned Library additions. An option to toggle this capability would be of utility. Having searched the manual (v2.6), we could only find entries for Analyze and BPM on p.107. This one would be a great time saver.

Auto DJ conditions to add newly scanned Library additions of a specified bpm to Auto DJ Crates. With rule for accommodating exact bpm mults/divisions e.g. 320bpm/160bpm/80bpm/40bpm are all added, but exclude new Library additions with different bpm from the Crates.

We don't know how Auto DJ or Mixxx in general looks under the hood yet. How would you envision the approach to extending Auto DJ to include the capabilities we are hoping for? What manageable steps would you recommend breaking it in to, and what of those steps might be low enough hanging fruit sized chunks that we could get to work on, or would it be better to first take a look at one of the other discreet features listed above?

Here's a 1080 video playing with generated material across 4 Decks:
https://www.youtube.com/watch?v=iJW8diFjkqY

[ronso0]

Auto-Library Rescan conditions...
Auto-Analyse newly scanned Library additions...
Auto DJ conditions to add newly scanned Library additions of a specified bpm to Auto DJ Crates

These sound useful, please file a feature request for each so it can be tracked. Thank you!

[daschuer]

would it be C++ for that work

Yes

Auto-Library Rescan conditions.

That will work. We may install optionanly a file system whatcher that automatically adds files from a certain location.

But I guess you actually need an "Inter Process Control" (IPC) interface, that allows you to add files to library or load to decks via command line, DBus, MPRIS, OSC or a raw network socket.

Can you file a new issue for this?

Auto-Analyse newly scanned Library additions.

Yes, Please file a separate issue for this?

Auto DJ conditions to add newly scanned Library additions of a specified bpm to Auto DJ Crates.

This sounds like an extention for the IPC interface

We don't know how Auto DJ or Mixxx in general looks under the hood yet.

You can look under the hood here:
https://github.com/daschuer/mixxx/blob/main/src/library/autodj/autodjprocessor.cpp

In general it follows the approch to behave like a real DJ would. So we may model new actions one by one into the auto DJ.
It is probably the next task to describe these actions in details.

[GomuSkelly]

Great. Thank you @daschuer and @ronso0

We will file those requests seperately, and take a look at autodjprocessor.cpp

[GomuSkelly]

@daschuer Here's a 1080 video playing with generated material across 4 Decks:

https://www.youtube.com/watch?v=iJW8diFjkqY

In this experiment, the AutoDJ plays material on Decks 1+2 generated using the prompt "160bpm birds trap dubstep jungle" which creates sparse rhythmic material with no melody, and some rhythmic birdsong.

Decks 3+4 are being DJ'd manually with material generated from both sonic texture prompts (e.g "160bpm birds"/"160bpm male vocals") and music prompts (e.g"160bpm trap heaven church dubstep jungle"), but in contrast to the playing style mentioned previously, in this instance the sonic texture also contains "160bpm" in the prompt which generates the sonic texture with rhythm.

Auto DJ conditions to add newly scanned Library additions of a specified bpm to Auto DJ Crates.

This sounds like an extention for the IPC interface

I reread your message today, apologies if I misunderstood you before filing #14143

Still reading through autodjprocessor.cpp. My partners's more familiar with C, but it's extremely well commented. Thanks to those involved for making life a lot easier for newcomers!

[GomuSkelly]

@daschuer Am I correct in thinking this is the section of autodjprocessor.cpp that we should be looking at adaptations for?:

    // TODO: This is a total bandaid for making Auto DJ work with four decks.
    // We should design a nicer way to handle this.
    for (const auto& pDeck : std::as_const(m_decks)) {
        VERIFY_OR_DEBUG_ASSERT(pDeck) {
            continue;
        }
        if (pDeck == pLeftDeck) {
            continue;
        }
        if (pDeck == pRightDeck) {
            continue;
        }
        if (pDeck->isPlaying()) {
            // Keep the current state.
            emitAutoDJStateChanged(m_eState);
            emit autoDJError(ADJ_UNUSED_DECK_PLAYING);
            return ADJ_UNUSED_DECK_PLAYING;
        }
    }

    if (pLeftDeck->index > 1 || pRightDeck->index > 1) {
        // Left and/or right deck is deck 3/4 which may not be visible.
        // Make sure it is, if the current skin is a 4-deck skin.
        ControlObject::set(ConfigKey("[Skin]", "show_4decks"), 1);
    }