FLUX第二弹！FLUX-GGUF量化，免费大碗的FLUX-API来了！

2024-08-21 1186

版权

本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。

简介： 近期GGUF 量化支持原生 ComfyUI 和FLUX模型啦。

引言

虽然量化不适用于常规 UNET 模型 (conv2d)，但 flux 等 transformer/DiT 模型似乎受量化的影响较小。这允许在低端 GPU 上以量化的方式运行它。近期GGUF 量化支持原生 ComfyUI 和FLUX模型。

模型链接：

https://modelscope.cn/models/AI-ModelScope/FLUX.1-dev-gguf

ComfyUI代码链接：

https://github.com/city96/ComfyUI-GGUF

因为FLUX对显卡有较高的要求，DashScope也推出了FLUX-API，支持中文输入，免费大碗，对于初体验FLUX的用户，也非常方便。

API文档链接：

https://help.aliyun.com/zh/dashscope/developer-reference/flux-apis

ComfyUI代码链接：

https://github.com/modelscope/comfyscope/

魔搭创空间：

https://modelscope.cn/studios/AI-ModelScope/flux

基于ComfyUI的FLUX最佳实践

除了在魔搭创空间上直接体验之外，这里我们提供基于ComfyUI的FLUX-GGUF模型和FLUX-API上手使用体验。ComfyUI是一个功能强大、模块化程度高的AIGC图形和视频生成的用户界面和后台。本文使用ComfyUI，在魔搭社区提供的免费GPU Notebook上，体验FLUX模型的使用：

下载和部署ComfyUI

clone代码，并安装相关依赖：

#@title Environment Setup
from pathlib import Path
OPTIONS = {}
UPDATE_COMFY_UI = True  #@param {type:"boolean"}
INSTALL_COMFYUI_MANAGER = True  #@param {type:"boolean"}
INSTALL_CUSTOM_NODES_DEPENDENCIES = True  #@param {type:"boolean"}
INSTALL_FLUX = True
INSTALL_FLUX_API = True  #@param {type:"boolean"}
OPTIONS['UPDATE_COMFY_UI'] = UPDATE_COMFY_UI
OPTIONS['INSTALL_COMFYUI_MANAGER'] = INSTALL_COMFYUI_MANAGER
OPTIONS['INSTALL_FLUX'] = INSTALL_FLUX
OPTIONS['INSTALL_FLUX_API'] = INSTALL_FLUX_API
OPTIONS['INSTALL_CUSTOM_NODES_DEPENDENCIES'] = INSTALL_CUSTOM_NODES_DEPENDENCIES
current_dir = !pwd
WORKSPACE = f"{current_dir[0]}/ComfyUI"
!pip install --upgrade gguf
%cd /mnt/workspace/
![ ! -d $WORKSPACE ] && echo -= Initial setup ComfyUI =- && git clone https://github.com/comfyanonymous/ComfyUI
%cd $WORKSPACE
if OPTIONS['UPDATE_COMFY_UI']:
  !echo "-= Updating ComfyUI =-"
  !git pull
# 安装FLUX-GGUF node
if OPTIONS['INSTALL_FLUX']:
  %cd custom_nodes
  !echo -= Initial setup FLUX =- && git clone https://github.com/city96/ComfyUI-GGUF
if OPTIONS['INSTALL_COMFYUI_MANAGER']:
  ![ ! -d ComfyUI-Manager ] && echo -= Initial setup ComfyUI-Manager =- && git clone https://github.com/ltdrdata/ComfyUI-Manager
  %cd ComfyUI-Manager
  !git pull
# 安装FLUX-API node
if OPTIONS['INSTALL_FLUX_API']:
  %cd ../
  ![ ! -d comfy-flux ] && echo -= Initial setup dashscope_api =- && git clone https://github.com/modelscope/comfyscope.git
  %cd comfy-flux
  !git pull
%cd $WORKSPACE
if OPTIONS['INSTALL_CUSTOM_NODES_DEPENDENCIES']:
  !pwd
  !echo "-= Install custom nodes dependencies =-"
  ![ -f "custom_nodes/ComfyUI-Manager/scripts/colab-dependencies.py" ] && python "custom_nodes/ComfyUI-Manager/scripts/colab-dependencies.py"

模型下载

#@markdown ###Download standard resources
%cd /mnt/workspace/ComfyUI
### FLUX1-DEV
!modelscope download --model=AI-ModelScope/FLUX.1-dev-gguf --local_dir ./models/unet/ flux1-dev-Q5_1.gguf
### clip
!modelscope download --model=AI-ModelScope/flux_text_encoders --local_dir ./models/clip/ t5xxl_fp16.safetensors
!modelscope download --model=AI-ModelScope/flux_text_encoders --local_dir ./models/clip/ clip_l.safetensors
!modelscope download --model=AI-ModelScope/flux_text_encoders --local_dir ./models/clip/ t5xxl_fp8_e4m3fn.safetensors
### vae
!modelscope download --model=AI-ModelScope/FLUX.1-dev --local_dir ./models/vae/ ae.safetensors

使用cloudflared运行ComfyUI

!wget "https://modelscope.oss-cn-beijing.aliyuncs.com/resource/cloudflared-linux-amd64.deb"
!dpkg -i cloudflared-linux-amd64.deb
%cd /mnt/workspace/ComfyUI
import subprocess
import threading
import time
import socket
import urllib.request
def iframe_thread(port):
  while True:
      time.sleep(0.5)
      sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      result = sock.connect_ex(('127.0.0.1', port))
      if result == 0:
        break
      sock.close()
  print("\nComfyUI finished loading, trying to launch cloudflared (if it gets stuck here cloudflared is having issues)\n")
  p = subprocess.Popen(["cloudflared", "tunnel", "--url", "http://127.0.0.1:{}".format(port)], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  for line in p.stderr:
    l = line.decode()
    if "trycloudflare.com " in l:
      print("This is the URL to access ComfyUI:", l[l.find("http"):], end='')
    #print(l, end='')
threading.Thread(target=iframe_thread, daemon=True, args=(8188,)).start()
!python main.py --dont-print-server