Give more customizable options for safety checker #815
Conversation
src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
Outdated
Show resolved
Hide resolved
|
The documentation is not available anymore as the PR was closed or merged. |
src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
Outdated
Show resolved
Hide resolved
src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
Outdated
Show resolved
Hide resolved
| " `from_pretrained`. For more information, please have a look at" | ||
| " https://github.com/huggingface/diffusers/pull/254" | ||
| ) | ||
|
|
There was a problem hiding this comment.
I think referencing another PR dilutes the message. I'd propose something like:
The
diffusersteam and Hugging Face strongly recommend to keep the safety filter enabled in all circumstances. Please, make sure you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public.
I think it's worthwhile to make this as clear as we can, maybe @natolambert, @mmitchellai, @yjernite can provide better wording.
There was a problem hiding this comment.
I would even say:
Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results.
I mean, I think proposing a use case when removing the safety filter is acceptable kind of justifies why we are proposing this PR to remove it.
| @@ -20,11 +20,11 @@ class StableDiffusionPipelineOutput(BaseOutput): | |||
| num_channels)`. PIL images or numpy array present the denoised images of the diffusion pipeline. | |||
| nsfw_content_detected (`List[bool]`) | |||
| List of flags denoting whether the corresponding generated image likely represents "not-safe-for-work" | |||
| (nsfw) content. | |||
| (nsfw) content. If safety checker is disabled `None` will be returned. | |||
There was a problem hiding this comment.
@patrickvonplaten one question/request -- would it be possible to have a setting that still returns whether or not an image is nsfw, but does not black out that image? This would allow devs who are building on top of this library to do something like show a popup to a user (e.g. 'you are about to view a NSFW image, do you want to proceed')
One possible way to implement this is to pass a flag into the safety checker module that disables the 'return a black image' part of the checker. Another way is to expose the safety checker class so that end users can add the checker in at the end (solely to get the bool[] indicating nsfw-ness)
Co-authored-by: Pedro Cuenca <[email protected]>
| f"You have disabed the safety checker for {self.__class__} by passing `safety_checker=None`.Please" | ||
| " make sure you have very good reasons for this and have considered the consequences of doing so.The" | ||
| " `diffusers` team does not recommend disabling the safety under ANY circumstances and strongly" | ||
| " suggests to not disable the `safety_checker` by NOT passing `safety_checker=None` to" |
There was a problem hiding this comment.
I don't think the second "by NOT passing safety_checker=None to from_pretrained." is necessary
* Give more customizable options for safety checker * Apply suggestions from code review * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py * Finish * make style * Apply suggestions from code review Co-authored-by: Pedro Cuenca <[email protected]> * up Co-authored-by: Pedro Cuenca <[email protected]>
* Give more customizable options for safety checker * Apply suggestions from code review * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py * Finish * make style * Apply suggestions from code review Co-authored-by: Pedro Cuenca <[email protected]> * up Co-authored-by: Pedro Cuenca <[email protected]>
This allows people to disable the safety checker. Note: This is never recommended in the eyes of the
diffusersteam, but we also don't want people to have to fork the repository in order to disable the safety checker if they deem it necessary in their eyes.For more information, please have a look at: #254 (comment) .
The safety checker can be disabled by doing: