mradermacher

/

maya1-GGUF

like 17

There are no multi-part GGUFs so no need to worry about concatenating them. Simply download and use them. To load GGUFs using ComfyUI you need ComfyUI-GGUF froim https://github.com/city96/ComfyUI-GGUF. Please follow the documentation of ComfyUI-GGU if you are unsure how to use it. We only test GGUFs using llama.cpp and so can’t tell you if this specific model is compatible with ComfyUI-GGUF.

can you help me for using it with llama.cpp? i mean about the prompts to use and load it?

You can just run it like .\llama-cli.exe -m .\maya1.Q5_K_M.gguf or more realisticaly you want to run .\llama-server.exe -m .\maya1.Q5_K_M.gguf so you can use the OpenAI API compatible server for your python script to interact with. Because the audio is done by generating special tokens no MMPROJ file is required. To conveart the generated tokens to actual audio you can use the python code provided under https://huggingface.co/maya-research/maya1:

# Decode SNAC tokens to audio frames
frames = len(snac_tokens) // 7
codes = [[], [], []]
for i in range(frames):
    s = snac_tokens[i*7:(i+1)*7]
    codes[0].append((s[0]-128266) % 4096)
    codes[1].extend([(s[1]-128266) % 4096, (s[4]-128266) % 4096])
    codes[2].extend([(s[2]-128266) % 4096, (s[3]-128266) % 4096, (s[5]-128266) % 4096, (s[6]-128266) % 4096])

# Generate final audio with SNAC decoder
codes_tensor = [torch.tensor(c, dtype=torch.long, device="cuda").unsqueeze(0) for c in codes]
with torch.inference_mode():
    audio = snac_model.decoder(snac_model.quantizer.from_codes(codes_tensor))[0, 0].cpu().numpy()

# Save your emotional voice output
sf.write("output.wav", audio, 24000)
print("Voice generated successfully! Play output.wav")

Female, in her 30s with an American accent and is an event host, energetic, clear diction
<custom_token_4><custom_token_3943><custom_token_5371><custom_token_11027><custom_token_15173><custom_token_20216><custom_token_20830><custom_token_25649><custom_token_449><custom_token_7408><custom_token_11939><custom_token_12550><custom_token_20087><custom_token_21730><custom_token_27538><custom_token_1475><custom_token_7184><custom_token_10741><custom_token_13447><custom_token_17216><custom_token_20815><custom_token_27948><custom_token_3653><custom_token_6718><custom_token_12009><custom_token_13641><custom_token_18527><custom_token_21393><custom_token_27048><custom_token_3129><custom_token_6260><custom_token_10722><custom_token_15776><custom_token_17065><custom_token_21380><custom_token_25711><custom_token_65><custom_token_7884><custom_token_8615><custom_token_15010><custom_token_17662><custom_token_23933><custom_token_24911><custom_token_2702><custom_token_7745><custom_token_12042><custom_token_15643><custom_token_17736><custom_token_22663><custom_token_26759><custom_token_165><custom_token_6501><custom_token_11304><custom_token_14023><custom_token_20414><custom_token_23540><custom_token_27804><custom_token_1351><custom_token_6707><custom_token_9818><custom_token_16224><custom_token_20085><custom_token_21934><custom_token_25988><custom_token_4088><custom_token_5323><custom_token_9173><custom_token_16271><custom_token_18223><custom_token_24126><custom_token_25300><custom_token_3159><custom_token_7123><custom_token_9589><custom_token_14474><custom_token_18223><custom_token_22085><custom_token_26500><custom_token_3159><custom_token_7430><custom_token_10232><custom_token_13770><custom_token_17057><custom_token_20650><custom_token_26431><custom_token_3159><custom_token_4137><custom_token_9818><custom_token_15027><custom_token_19718><custom_token_24418><custom_token_26616><custom_token_1696><custom_token_4474><custom_token_10181><custom_token_14431><custom_token_19791><custom_token_20705><custom_token_24999><custom_token_3840><custom_token_7453><custom_token_9608><custom_token_12661><custom_token_19899><custom_token_23019><custom_token_26661><custom_token_3653><custom_token_6023><custom_token_11092><custom_token_13719><custom_token_18248><custom_token_24310><custom_token_28291><custom_token_2253><custom_token_5374><custom_token_8942><custom_token_15990><custom_token_17780><custom_token_23821><custom_token_27573><custom_token_586><custom_token_4639><custom_token_11510><custom_token_13418><custom_token_19354><custom_token_23635><custom_token_24673><custom_token_832><custom_token_4470><custom_token_9903><custom_token_16228><custom_token_19403><custom_token_20686><custom_token_25511><custom_token_2170><custom_token_6360><custom_token_8241><custom_token_15400><custom_token_17494><custom_token_24019><custom_token_28278><custom_token_2702><custom_token_7364><custom_token_11795><custom_token_12972><custom_token_19607><custom_token_24211><custom_token_26375><custom_token_65><custom_token_6985><custom_token_8528><custom_token_15603><custom_token_19607><custom_token_22151><custom_token_24732><custom_token_322><custom_token_4870><custom_token_9545><custom_token_13948><custom_token_19078><custom_token_21816><custom_token_26328><custom_token_3101><custom_token_6435><custom_token_9099><custom_token_15606><custom_token_19443><custom_token_21584><custom_token_24738><custom_token_832><custom_token_5173><custom_token_11456><custom_token_15870><custom_token_18139><custom_token_24093><custom_token_25519><custom_token_586><custom_token_7358><custom_token_8209><custom_token_13587><custom_token_18503><custom_token_20588><custom_token_24593><custom_token_3845><custom_token_6249><custom_token_11824><custom_token_13090><custom_token_17790><custom_token_23106><custom_token_27066><custom_token_3845><custom_token_5502><custom_token_9545><custom_token_13437><custom_token_19373><custom_token_22565><custom_token_27454><custom_token_1098><custom_token_7909><custom_token_8594><custom_token_15485><custom_token_16723><custom_token_23208><custom_token_25872><custom_token_3694><custom_token_4627><custom_token_8908><custom_token_12847><custom_token_19079><custom_token_22875><custom_token_26490><custom_token_1244><custom_token_7495><custom_token_8295><custom_token_16104><custom_token_16943><custom_token_22948><custom_token_27142><custom_token_3653><custom_token_5348><custom_token_12232><custom_token_12444><custom_token_17662><custom_token_23990><custom_token_26893><custom_token_385><custom_token_5519><custom_token_10106><custom_token_16385><custom_token_19496><custom_token_23951><custom_token_26071><custom_token_1098><custom_token_4236><custom_token_9261><custom_token_15798><custom_token_17400><custom_token_22992><custom_token_25169><custom_token_466><custom_token_5941><custom_token_10453><custom_token_13654><custom_token_20470><custom_token_24296><custom_token_27643><custom_token_2723><custom_token_8109><custom_token_8420><custom_token_13016><custom_token_19851><custom_token_21934><custom_token_27804><custom_token_2253><custom_token_6130><custom_token_9087><custom_token_12316><custom_token_19233><custom_token_23951><custom_token_27397><custom_token_1093><custom_token_6566><custom_token_9034><custom_token_13275><custom_token_19886><custom_token_20715><custom_token_25476><custom_token_762><custom_token_7477><custom_token_10703><custom_token_15936><custom_token_19899><custom_token_22335><custom_token_26902><custom_token_2224><custom_token_4408><custom_token_11576><custom_token_12361><custom_token_16505><custom_token_22649><custom_token_26419><custom_token_3610><custom_token_7358><custom_token_9846><custom_token_13902><custom_token_17289><custom_token_22441><custom_token_26021><custom_token_865><custom_token_7297><custom_token_8885><custom_token_12869><custom_token_18082><custom_token_20595><custom_token_26241><custom_token_1093><custom_token_7643><custom_token_11700><custom_token_14168><custom_token_18496><custom_token_23303><custom_token_26973><custom_token_3845><custom_token_6131><custom_token_9233><custom_token_12337><custom_token_17929><custom_token_24337><custom_token_26629><custom_token_2702><custom_token_7670><custom_token_8885><custom_token_12715><custom_token_18809><custom_token_23589><custom_token_26823><custom_token_1379><custom_token_6807><custom_token_10756><custom_token_16361><custom_token_16700><custom_token_22257><custom_token_27389><custom_token_3406><custom_token_4379><custom_token_9031><custom_token_15051><custom_token_18579><custom_token_21233><custom_token_28569><custom_token_1768><custom_token_4348><custom_token_9926><custom_token_16165><custom_token_17788><custom_token_24268><custom_token_26628><custom_token_4086><custom_token_4162><custom_token_12271><custom_token_13348><custom_token_17082><custom_token_21299><custom_token_28476><custom_token_3049><custom_token_6215><custom_token_10079><custom_token_12794><custom_token_19196><custom_token_24356><custom_token_24734><custom_token_409><custom_token_5657><custom_token_10988><custom_token_13546><custom_token_18377><custom_token_24044><custom_token_26972><custom_token_3848><custom_token_5025><custom_token_11142><custom_token_16016><custom_token_18078><custom_token_23247><custom_token_24811><custom_token_3159><custom_token_4417><custom_token_10486><custom_token_14949><custom_token_18466><custom_token_24432><custom_token_26758><custom_token_762><custom_token_6178><custom_token_9869><custom_token_15358><custom_token_17359><custom_token_21238><custom_token_24739><custom_token_1379><custom_token_4592><custom_token_11648><custom_token_12927><custom_token_16723><custom_token_20606><custom_token_27298><custom_token_2224><custom_token_4435><custom_token_8213><custom_token_13100><custom_token_17947><custom_token_23744><custom_token_26353><custom_token_2869><custom_token_4987><custom_token_9105><custom_token_16154><custom_token_17844><custom_token_20536><custom_token_26893><custom_token_3634><custom_token_5364><custom_token_12293><custom_token_14685><custom_token_17313><custom_token_24473><custom_token_28205><custom_token_1379><custom_token_5216><custom_token_10424><custom_token_14767><custom_token_20485><custom_token_21996><custom_token_25548><custom_token_1379><custom_token_4435><custom_token_10008><custom_token_13319><custom_token_19886><custom_token_20758><custom_token_27792><custom_token_1379><custom_token_5721><custom_token_9806><custom_token_13217><custom_token_17368><custom_token_22772><custom_token_25334><custom_token_3848><custom_token_5313><custom_token_8751><custom_token_13654><custom_token_18151><custom_token_22442><custom_token_27103><custom_token_3159><custom_token_6545><custom_token_10526><custom_token_13214><custom_token_19944><custom_token_22004><custom_token_26222><custom_token_3159><custom_token_5039><custom_token_9869><custom_token_15456><custom_token_19038><custom_token_23765><custom_token_26253><custom_token_3159><custom_token_4137><custom_token_9958><custom_token_14470><custom_token_17783><custom_token_24530><custom_token_25573><custom_token_1692><custom_token_4769><custom_token_12135><custom_token_15861><custom_token_20487><custom_token_20811><custom_token_27910><custom_token_832><custom_token_6217><custom_token_8328><custom_token_12745><custom_token_17129><custom_token_22168><custom_token_26624><custom_token_3236><custom_token_6241><custom_token_8326><custom_token_14432><custom_token_16728><custom_token_21040><custom_token_27730><custom_token_50><custom_token_6468><custom_token_9122><custom_token_12797><custom_token_17047><custom_token_22341><custom_token_27937><custom_token_1820><custom_token_6578><custom_token_9583><custom_token_13195><custom_token_19080><custom_token_20636><custom_token_25505><custom_token_466><custom_token_4720><custom_token_8354><custom_token_15530><custom_token_16804><custom_token_24203><custom_token_26538><custom_token_419><custom_token_5464><custom_token_8690><custom_token_14854><custom_token_17440><custom_token_23015><custom_token_28093><custom_token_2224><custom_token_7889><custom_token_11349><custom_token_12981><custom_token_20089><custom_token_22134><custom_token_28564><custom_token_1093><custom_token_5519><custom_token_10575><custom_token_13344><custom_token_17348><custom_token_21235><custom_token_26518><custom_token_612><custom_token_5976><custom_token_8987><custom_token_15370><custom_token_18130><custom_token_23394><custom_token_26441><custom_token_2><custom_token_6><custom_token_3> When people want to meet her, they come to her for advice.

could you please provide a the file for loading the model & inference? thx

could you please provide a the file for loading the model & inference? thx

You simply use llama-server to host the model and do inference. You then create a simple python script that communicates with the OpenAI compatible llama-server to obtain the audio token which it then can converts into a WAV file or directly play by adapting the python code above.

Worth mentioning, depending on how you serve the GGUF, make sure to disable the embedded Jinja template, as it will completely garble your output.

Worth mentioning, depending on how you serve the GGUF, make sure to disable the embedded Jinja template, as it will completely garble your output.

How do I do that using llama-server?

Worth mentioning, depending on how you serve the GGUF, make sure to disable the embedded Jinja template, as it will completely garble your output.

How do I do that using llama-server?

I don't use llama-server personally, but using the /completion endpoint should avoid prompt formatting, allowing the custom prompt demonstrated in their reference code to work just fine.

I'd like to use a GGUF of Maya1 with this repo: https://github.com/PkmX/orpheus-chat-webui, which is a cool STT and TTS chatbot designed for use with orpheus. However, I'm not sure how to incorporate the Maya code listed above to make the switch. Any suggestions would be most appreciated!

I'd like to use a GGUF of Maya1 with this repo: https://github.com/PkmX/orpheus-chat-webui, which is a cool STT and TTS chatbot designed for use with orpheus. However, I'm not sure how to incorporate the Maya code listed above to make the switch. Any suggestions would be most appreciated!

It looks doable. Looking at the repo, specifically this file: https://github.com/PkmX/orpheus-chat-webui/blob/main/src/orpheus-chat-webui/__main__.py

Line 22: Tweak to point to the endpoint serving Maya1.

Line 80: Tweak to match the prompt format in the reference script provided for Maya1.

You'll need to make sure the voice string is a description, per Maya1, but both models use SNAC, and use custom tokens, so that might be all that is needed (if you are lucky).

If you are not lucky, the token alignment will be off, but since both models are using SNAC under the hood, the total number of SNAC tokens should be the same, which means you'd need to look at the tokenizer.json files for each model to figure it out, and apply a tweak to extract_custom_tokens at line 95 (basically adding or subtracting an offset to fix the alignment).

Some of the code (setting up the tokens for the prompt, for instance) can just be lifted from the Maya1 reference code, and I could be totally wrong about the tokens (I'm just assuming that because both models use SNAC under the hood that things will line up nicely), and verifying that on a whim is more spare time than I have now, but that should get you pointed in the right direction.

at line 80, I tried:
def generate_mayaone_tokens(
prompt: str, voice: str = "tara"
) -> Generator[str, None, None]:
response = mayaone.completions.create(
model="mayaone",
description="30-year-old female, professional, conversational",
prompt=f"<|audio|>{voice}: {prompt}<|eot_id|>", stream=True
)

but that doesn't work. should the "description" go elsewhere?

The whole prompt construction needs to follow the example for Maya1:

formatted_text = f'<description="{description}"> {text}'

prompt = (
    soh_token + bos_token + formatted_text + eot_token +
    eoh_token + soa_token + sos_token
)

return prompt

So the description, which you have defined, would do where the {voice} is, but the rest of that prompt will also not work, because it is not formatted like Maya1 is expecting.

If you have access to a coding LLM, you could probably feed it both scripts, and ask it to fold in the Maya1 code, because there are going to be some more details to get right, like loading the tokenizer for Maya1.