new baseconfig setting that aworks in router mode

[Rose22]

implements a new --baseconfig commandline setting. it's intended for admin mode and router mode.

In router mode (the main usecase), it makes it so that when you switch between kcpps files/configs, it applies those configs ON TOP of the baseconfig you specified. It does that by applying those configs as override configs on top of the base config, instead of loading them as the main config.

In admin mode, it serves as a convenient default, where if you load a kcpps file using koboldlite's dialog without specifying a base config (so if you just specify one config file), it loads that on top of the base config (so here too it turns it into an override config). But you can still select your own primary and secondary config and it will still respect that and allow you to manually select a different primary and secondary config :)

this makes router mode very useful because you can now create a base config file containing all your usual koboldcpp settings like what gpu backend to use, how many gpu layers, embedding models, tts models, and so on, and then you can have individual model config files that contain only the settings that are needed for specific models!

[Rose22]

btw i extensively tested it on my own setup, made sure it actually works. i wonder if it will work the same for you?

[LostRuins]

I added the GUI fields portion. I also did a small change - I don't load the baseconfig file, on the very first launch, because that corrupts the original launch args and also breaks config loading in GUI launcher mode.

Instead, I simplified it a bit. baseconfig will simply be the default for the override_config during any model swap loading. The logic will be:

if new_target and override_config is both specified:
  load override_config, then load new_target on top. base_config is not used.

if new_target and override_config is not specified, but base_config is specified:
  load base_config, then load new_target on top

if new_target and override_config is not specified, and base_config is not specified:
  load new_target only.

then this will be more accurately reflected in the GUI too

Very clear now - either you select an override and it overrides, or you do not, and it does the baseconfig.

And if you're using the OpenAI routermode, then its always applying the new model on top of your defined baseconfig (as there's never any override in OAI mode), just as you wanted.

[LostRuins]

I also added the field ignorelist at the proper location. These are the new ignorefields which will never be modified during a model swap.

["remotetunnel","showgui","port","host","port_param","admin","adminpassword","password","adminunloadtimeout","routermode","admindir","ssl","nocertify","benchmark","prompt","config","baseconfig","downloaddir"]

[Rose22]

most of that seems good, but by swapping the order back around, i think you've broken it:

elif targetfilepath and targetfilepath2 and targetfilepath2!="":
    # stuff applied later overwrites stuff applied earlier, if they have the same field names
    reload_from_new_args(defaultargs)
    reload_new_config(targetfilepath2,vars(args),True)
    reload_new_config(targetfilepath,vars(args),True)

try setting useswa or contextsize in a config that's not the baseconfig (modelconfig.kcpps in case of --baseconfig base.kcpps --config modelconfig.kcpps) and watch it not apply. it won't function in its current state because with the new --baseconfig argument it assumes it's the base config and that it's values are to be overridden by the override config. if it's swapped, it instead tries to override non existent values in the override config with base config keys/values, which don't exist in the override configs so it just silently fails and skips it

if new_target and override_config is not specified, but base_config is specified:
  load base_config, then load new_target on top

i see your logic. if this was actually the case, it wouldn't break

but the actual code is:

#if override config is not specified, AND baseconfig is, swap it as our override
if restart_target!="unload_model" and restart_target!="initial_model" and args.baseconfig and not targetfilepath2:
    basecfg_path = os.path.abspath(args.baseconfig)
    if os.path.exists(basecfg_path):
        targetfilepath2 = basecfg_path
        print(f"No override config provided, using baseconfig {args.baseconfig}")

which loads the base config on top of the normal config, breaking everything. the base config is supposed to be the bottom layer, with the other configs being loaded on top of it

and as for the way you're loading override configs.. that doesn't make much sense to me either, why call them override configs if they are instead the configs that get overridden by your selected config?

[LostRuins]

Base config and override config are supposed to be referring to the same thing. I should rename the field because it's confusing now.

We have 2 fields. We have the main_target and the base_config.
Idea: The base_config is applied first, then main_target is applied, the main_target overwrites any conflicting fields.

Your original request: For OAI endpoints, allow base_config to work - Since OAI only allows one single main_target, then base_config is to be supplied by a file instead of by API .

---

Expanding your request:
Step 1. Loading --config modelconfig.kcpps in the GUI
Step 2. Setting the field --baseconfig base.kcpps

Step 3. Launching koboldcpp. Now the session is ready.
Step 4. Invoke a model change to hello.gguf with some_base.kcpps provided over API

• some_base.kcpps settings are loaded
• hello.gguf is applied on top of some_base.kcpps
• model swap completed
  Step 5. Invoke a model change to hello.gguf, no base provided (default).
• base.kcpps settings are loaded (since its not specified in API)
• hello.gguf is applied on top of base.kcpps
• model swap completed.

so where is it broken?
base config <- this contains all your defaults
new target <- this overrides the base

[LostRuins]

I think maybe you are expecting the base config to be applied immediately after launch even if no model swap if triggered? but wouldn't that instantly overwrite everything you have set in a config, a GUI, or even launch params?

e.g.
koboldcpp.exe --model gimme_apples.gguf --baseconfig oranges.kcpps <- apples are worthless if baseconfig applies at the start!

[LostRuins]

I got rid of all references to overrideconfig to reduce confusion 45737ef, and i swapped the fields in the GUI