AI character consistency: hosted tools or your own LoRA

Consistency is three problems wearing one name

An image can look exactly like your persona and still feel like a different person from yesterday’s post. Most people then assume the training failed and go straight back to the trainer. Often that is the wrong fix. Before you retrain anything, find out which of three layers broke.

• Structure is the shape of the face: skull width, jaw angle, eye spacing, nose bridge, the distance from lip to chin. A LoRA?LoRA A LoRA is a small file trained on images of one subject. It teaches a general image model to reproduce that specific face or style. Open the glossary learns this and can hold it. If the proportions change between images, you have a training problem.

• Surface is skin texture, pore density, colour temperature, how shiny the highlights are, and how the upscaler renders hair. Two images with the same face shape and a different surface look like two different people shot on two different cameras. Your LoRA has almost nothing to do with this. It comes from the sampler, the scheduler, the step count, the guidance value, the upscale model and the denoise of the detail pass.

• Signature is everything outside the face that people use to recognise someone in a small thumbnail. That means hair length and parting, a mole, ear shape, body proportions, and the few lighting and lens words you repeat in every prompt. At the size of a phone feed, signature does more for recognition than structure.

So check in a fixed order. Open two images that feel like different people and ask three questions. Are the proportions different (structure)? Is the skin and texture different (surface)? Did the hair or the framing change (signature)? You only go back to training if the answer to the first question is yes.

1. Datasetcasting sheet, not album
2. TrainFlux LoRA
3. Sample setfixed seeds and prompts
4. Drift checkside by side, epoch by epoch

Two images feel like different people. What changed?

The proportionsThe skin and materialHair and framing

• StructureBone geometry. The one layer a LoRA learns: back to the dataset, then retrain.
• SurfaceSampler, scheduler, steps, guidance, upscaler, detail-pass denoise. Fix them as constants. You do not need a training run for this.
• SignatureThe recognisable non-facial details and the lighting words. Pin them in the prompt.

Schematicthe shape, not measured data

Hosted tools or your own LoRA: which to start with

There are two ways to keep a face the same. They cost very different things. Most people in this niche only talk about the harder one.

• Hosted models (reference-conditioned). You upload reference images and the model uses them every time it generates. Higgsfield and WaveSpeed both work this way, with the current model families: Seedance, Wan, MiniMax, Seedream, Nano Banana, Flux?Flux Flux is a family of image-generation models from Black Forest Labs. It is widely used for photorealistic characters because it holds faces better than earlier open models. Open the glossary. ChatGPT works the same way: ChatGPT AI influencer photos shows which reference images to upload and how to keep the face. Seedance is the exception for faces: Seedance consistent character explains why it refuses realistic faces in references. It takes minutes, where training takes hours. You need no GPU and no install. It is also the only route that works well for video today. Which models hold a face once it moves covers what each costs. To copy movement from a real video and keep her face, Kling motion control shows how to bind a face element. You are renting someone else’s pipeline. What that costs per image is on what generation costs.

• A LoRA you train and own. The weights are on your own disk. You need hours of setup, a rig or rented compute, and you have to learn a lot. In return you can move the face between workflows, you pay nothing per image, and nobody can take it away when a product changes its terms.

Start hosted. First find out whether you can build an audience. Keeping a face steady with your own LoRA is a separate skill, and you can learn it later. Almost everyone who quits does so long before a LoRA would have mattered.

If you start on Higgsfield, do not plan to train that later LoRA on its images. Its terms forbid training any model on its outputs:

Last updated 26 July 2026. Section 5.2(iv): you may not “use the Service, any Outputs, or any data derived therefrom to train, fine-tune, distill, or otherwise transfer knowledge to any machine-learning model, neural network, or artificial intelligence system, whether or not such system competes with the Service, except: (a) models that you create, train, or fine-tune using the Service with your own Inputs (not Outputs of Company’s AI models); or (b) as expressly authorized in writing by Company”. Section 5.1(iii) says the same for products “that compete with the Service”.

Higgsfield — Terms of Use Agreement (opens in a new tab)checked 5 Oct 2026

So a LoRA dataset made of her Higgsfield images is not allowed without Higgsfield’s written permission. Build the images for a future LoRA with a tool whose terms allow it.

Move to a trained LoRA only when one of these is true:

• The hosted tool keeps pulling the face towards its own standard face, and your character starts to look generic.
• You generate so many images that the per-image price is the biggest cost in your budget.
• You need a specific angle, expression or condition that the hosted tool does badly, and no prompt fixes it.
• You want the character to still exist if the product you built her in goes away.

The rest of this guide covers both routes. The dataset and diagnosis sections apply to both. A bad set of reference images breaks a hosted tool just as badly as it breaks a trained LoRA.

How to build a LoRA training dataset

The dataset matters more than all the training settings together. For a face LoRA on Flux, use twenty to forty images that differ from each other. More than that usually just adds repeats. Repeats do harm, because the model learns that whatever repeats is part of the identity. What each trainer vendor publishes as a number, and why they disagree, is in how many images to train a character LoRA.

Check your own set against this list before you train:

Does your set cover all five areas?

• Angles: front, three-quarter left, three-quarter right, and at least two images with the head tilted up or down.If 90% of the set is frontal, the LoRA loses the face as soon as your persona turns her head.
• Expressions: neutral, closed smile, open smile with teeth, mouth open mid-sentence, narrowed eyes.Most beginner datasets have too few expressions.
• Lighting: soft window light, hard direct sun, warm indoor light, flat overcast light.If every image has the same main light, the LoRA learns that light as part of the face.
• Camera distance: roughly half head-and-shoulders, a quarter mid-shot, a quarter full body.This decides whether people still recognise the face at feed size.
• Backgrounds: different in each image.If they are all the same, the background becomes part of the token.

These things ruin a set:

• Near-duplicates. Burst frames a third of a second apart look different to you. For training they count as the same image.
• A repeated accessory. If the same hoop earrings are in thirty images, the LoRA puts hoop earrings on your persona in a swimming pool. The same goes for a necklace, a specific top or a watch.
• Another face in the frame. Even blurred, even in a mirror, even at the edge. Crop it out or drop the image.
• Low-resolution sources that were upscaled. If you train on 512px crops upscaled to 1024, the LoRA learns the upscaler’s artefacts as skin texture.
• A set generated from one seed?Seed A seed is the number that sets the random starting point of a generation. The same prompt with the same seed produces the same image. Open the glossary by the base model. This one is easy to miss. If you made your reference images as fifty variations of one seed, you train on the model’s own average face. The LoRA then sharpens that average face and does not lock in one identity.

On Flux you write captions in normal sentences, not as a list of tags. Give the person a rare token. Then describe everything that varies: pose, clothing, background, light, camera distance. Do not describe the face. Anything you describe in the caption, you can still change later in the prompt. What you leave out of the caption gets tied to the token. This explains most of the “my LoRA always generates the same shirt” complaints you will read.

Training a Flux LoRA: the settings that matter

Everything from here to the diagnosis section is about the second route. If you use hosted models, skip to the ten-minute check. The same things go wrong there, even though the settings are different.

Pick one trainer and stay with it. The common choices for Flux LoRAs are kohya’s sd-scripts, ostris’s ai-toolkit and SimpleTuner. You cannot copy settings from one trainer to another. The same learning rate behaves differently in each one, because the optimiser defaults, the loss weighting and the blocks each trainer touches are different. Copying a number from a Reddit comment written for a different trainer is the most common reason a run produces a useless, blurry LoRA.

Before you build anything commercial on Flux, check the licence of the model you train on:

FLUX.1 dev weights are published under a non-commercial licence. Output may be used “for any purpose (including for commercial purposes)”, but using the model itself for revenue-generating activity is outside the non-commercial licence and requires a separate licence from Black Forest Labs. The schnell variant is Apache-2.0. Check the current terms on the licence page yourself before monetising anything trained on dev — this is exactly the kind of term that gets revised, and “someone said it was fine” is not a defence.

Black Forest Labs — FLUX [dev] Non-Commercial License v2.0 (opens in a new tab)checked 15 Sept 2026

These settings change the result, the most important first:

1. Total steps relative to dataset size. Total optimiser steps are roughly images times repeats times epochs, divided by batch size. There is no correct number. It depends on your image count, your learning rate and how distinctive the face is. Find your number by sampling. Do not copy one. Save a checkpoint every epoch.

2. Learning rate. People usually quote a rate around 1e-4 with an 8-bit AdamW optimiser as the starting point for Flux LoRAs. Check the default in the example config that ships with your trainer. Do not trust a forum post. The defaults have changed between releases, and only the config in the repository you installed is reliable. If you halve the rate and roughly double the steps, you get a gentler LoRA that works better together with other conditioning.

3. Rank. Rank 16 is enough for a human face. Rank 32 gives some more detail and a clearly higher chance of overfitting. Above 64 for a single face, you mostly waste parameters and ship a bigger file.

4. Resolution and bucketing. Train at the resolution you generate at. If you make 1024px images and train at 512, the model has to guess detail it never saw.

5. Text encoders. Leave T5 and CLIP frozen for a face LoRA. Training them changes much more than people expect. The model follows prompts worse long before the likeness gets better.

The most useful result of a training run is a comparison grid, even more than the final safetensors file. The grid shows the same six prompts with the same fixed seed, rendered from every saved checkpoint. Without it you are guessing which checkpoint is best.

The ComfyUI workflow that stops face drift

The structure matters more than the exact nodes, because node packs are renamed and forked all the time. There are four stages, in this order:

• Base generation. Your checkpoint, your LoRA at strength 0.85 to 1.0, at your working resolution. Nothing special here.

• Face detection and detail pass. A detector node finds the face. The face is cropped, rendered again at a much higher effective resolution with the same LoRA, and then put back into the image. In practice the denoise for that pass is around 0.35 to 0.45. Go higher and the pass invents a new face that no longer fits the body. Go lower and it only smooths. This one stage removes most of the drift?Drift Drift is the slow change in a character’s face over time, until images made weeks apart no longer look like the same person. Open the glossary that comes from camera distance. So when a LoRA fails in full-body shots, “train longer” is almost never the answer.

• Upscale last. If you upscale before the detail pass, you spend compute on a face the detail pass is about to replace.

• Fixed inference settings across the whole account. Sampler, scheduler, step count, guidance and upscale model become constants. Write them down. This is the surface layer from the first section. You fix it with one decision, without a training run.

Two habits that save you time later:

• Keep one workflow JSON per persona under version control. Put the workflow name and version into the output filename. When an image from three weeks ago looks wrong, you need to know which graph made it.
• Do not add a step at the end that strips metadata or makes a “clean export”. Generators embed provenance?Provenance Provenance is the recorded history of how a piece of media was made and changed. Open the glossary markers, and platforms read them to label your posts. See Don’t strip your watermarks and the EU compliance guide.

Trained LoRA or reference tools like Soul ID

There are two families of tools. Each does a different job.

• Trained identity. A LoRA you own, with the weights on your own disk. It takes hours of setup. It costs nothing per image, you can move it between workflows, and you can retrain or fine-tune it later. Use this for your persona’s main face.

• Reference-conditioned identity. Adapters in the IP-Adapter FaceID and PuLID family, and hosted products built on the same idea. Higgsfield’s Soul ID is the best-known example in this niche. You upload reference images and the model uses them every time it generates. It takes minutes instead of hours and needs no GPU rig. It also works for video, where LoRA pipelines are much harder.

All reference-conditioned tools have the same downside: they pull the face towards their own standard face. Faces come out slightly more symmetrical, slightly more conventionally attractive, and slightly more like everyone else’s output from the same tool. In a single campaign nobody sees it. For a face that has to stay the same person for a year, the small changes add up.

Most stable accounts split the work like this. The LoRA makes the main face. A reference-conditioned tool handles the cases the LoRA does badly. Then every image goes through the same detail pass and the same upscaler, so the surface matches wherever the image came from.

If you are looking at Soul ID or a similar hosted product, check four things on the vendor’s own page on the day you decide. Do not rely on a review.

• how many reference images it takes, and whether it keeps them
• whether you can export anything you could still use if the product disappeared
• what the commercial-use terms say
• whether outputs carry provenance metadata, and whether the terms forbid removing it (you want to keep it, so you need to know it is there)

Pricing and limits in this category change every month. Any number in a guide may be out of date by the time you read it.

Which model for which consistency job

Choose the method by the job, then the model. For stills, start with reference images and move to a LoRA when the face has to be yours for years. For video, use reference images or a saved element when you describe the scene in words. Use motion control when the movement has to be real.

There are four methods. Each holds something different, and each fails in its own way:

• Reference images. You upload a few images of her, and the model looks at them every time it generates. You need no training. The face holds roughly and drifts towards the tool’s standard face.
• A LoRA. Weights trained on her face. It holds the face shape best, and you own it. You need a dataset, a training run and a model that can load it.
• A saved element. A named set of reference images that a platform stores for you. Every clip uses the same set, so clip ten starts from the same information as clip one.
• Motion control. A video of a real person drives her movement. It copies the movement, not the face. The face still comes from your image and, in Kling, from a face element you bind.

Pick the job you have right now:

What do you need to keep the same?

Her face in stills, starting todayHundreds of stills a monthA clip from a written sceneThe same face in every clipA real movement, like a danceA face the model refuses

• Reference imagesUpload three to five strong images to a hosted tool. No training. Expect a slight pull towards the tool's standard face.
• Your own LoRATrain once and pay nothing per image. The face is yours, and you can take it to any workflow.
• Reference-to-videoGive the video model several images of her, up to nine in MiniMax H3. Include side views if she turns her head.
• A saved elementStore her reference set once, as a Higgsfield Element or a Kling element, and use it for every clip.
• Motion controlFilm the movement yourself and let Kling 3.0 copy it onto her. Bind a face element and keep the orientation on match video.
• Another model for her faceSeedance blocks realistic faces in its references. Use it for shots without her face, and a model that takes your references for the rest.

Schematicthe shape, not measured data

Stills: the images every video model starts from

The earlier sections give the rule for stills: start with a hosted tool and move to a LoRA when one of the four signs appears. One more reason to get the stills right comes from video. Every video method above starts from your stills. A start frame, a reference set, a saved element and a motion control image are all stills of her.

So a weak still set shows up in every clip. If your LoRA makes twenty clean, varied images of her, every video model gets a better reference set than from ten lucky outputs of a hosted tool. Build the set with the five areas from the dataset checklist: angles, expressions, light, distance and background.

Video: what each method holds and where it breaks

Method	What it holds	Where it breaks
Start frame (image-to-video)	Her face in the first frame	Towards the end of the clip, and when she turns her head
Reference-to-video	Her face from several images, in any scene you describe	At angles you did not upload, and on models that refuse realistic faces
Saved element	The same reference set in every clip	It is only as good as the images in it
Motion control	A real movement, with her face from a bound element	Fast moves, cuts, two people in the take, or the wrong orientation setting
LoRA in video	The face shape you trained	Only a few video models and hosts load one

Reference-to-video gives you the most room. You describe any scene, and the model checks it against several images of her:

Reference limits: “Images: ≤ 9; width/height in [256, 5760]”, “Videos: ≤ 3 clips; per-clip duration [2, 15] s; total duration ≤ 15 s”, “Mixed input is capped at 12 files in total.”

MiniMax API docs — Video generation guide (opens in a new tab)checked 5 Oct 2026

A saved element fixes a different problem. A reference you attach to one generation helps only that one generation. Higgsfield says so for Seedance:

“@character references hold a face consistent within a single generation. For the same character across multiple separate generations, save it as an Element”.

Higgsfield Help — How do I use Seedance (opens in a new tab)checked 5 Oct 2026

Motion control has its own limit. Kling’s element holds only the face, and only in one orientation mode:

“The Motion Control Element Library only uses facial information for reference. It does not include clothing, hairstyle, makeup, or props.” “Element Binding is supported only when the character’s orientation matches the video orientation.”

Kling Quickstart — Motion Control user guide (opens in a new tab)checked 5 Oct 2026

So her hair and outfit come from your image and your prompt. Keep both the same across clips, the same way you pin the signature layer in stills.

A LoRA in video is the most direct way to carry a trained face into motion. Few video models load one, and the hosts that do charge their own price per clip. Which models hold a face once it moves lists the ones that take a LoRA, and what generation costs has the current prices.

When a model refuses her face

Some video models check your references for realistic faces and refuse them. For a photorealistic persona, that can end a plan before the first clip.

“Dreamina Seedance 2.5 and Dreamina Seedance 2.0 series models do not support directly uploading reference images or videos that contain real human faces.”

BytePlus ModelArk — Create a video generation task (opens in a new tab)checked 5 Oct 2026

BytePlus names no exception for a face you generated in another tool. Test this before you build a pipeline on any model. Send one short clip with her front close-up through the host you plan to use. If it is refused, give her face shots to a model that takes your references, and keep the strict model for shots without her face.

A setup that covers most accounts

1. Make the stills with a hosted reference tool or your LoRA.
2. Pick five of them as her reference set: front, both three-quarter views, one full body, one in different light.
3. Save the set as an element wherever the platform allows it, so every clip uses the same images.
4. Use reference-to-video for scenes you describe, and motion control for movements you film.
5. Check the first and last frame of every clip against your master still, and run the monthly grid below on the stills.

Video models change often. Your reference set and your checks carry over to the next one.

The ten-minute check that catches drift early

Build a fixed grid of twelve prompts: frontal portrait, three-quarter, profile, full body, indoor, outdoor, hard light, soft light, smiling, neutral, and two of whatever your account posts most. Fix the seeds. Save the grid as your baseline.

Run it again every month, and after every change to the checkpoint, the LoRA, the sampler, the upscaler or any node pack. Put the new grid next to the baseline. You almost never see drift from one image to the next. You see it clearly when you put images from different months side by side. Your audience sees your images across months. You normally do not compare them that way, so the grid does it for you.

Numbers in this guide to check yourself

Model versions, trainer defaults and node names change faster than any written guide. Before you rely on a value from this guide, check:

• The example config that ships with your trainer, for the current default learning rate, rank and optimiser, in the version you installed.
• The Flux licence terms on Black Forest Labs’ own page, for anything commercial.
• The current node names in your ComfyUI?ComfyUI ComfyUI is a node-based interface for building image-generation pipelines. Each step is a box that is wired to the next. Open the glossary custom node packs. Detector and detailer nodes get renamed across major versions.
• The vendor page of any hosted identity tool, for limits, export and provenance behaviour.

Some numbers in this guide are ranges on purpose. A wrong exact number costs you more time than a range.

There is more in Character consistency. For the labelling side of a photorealistic persona, read Instagram’s AI-generated profile label.