• BaroqueInMind@lemmy.one
      link
      fedilink
      English
      arrow-up
      15
      ·
      edit-2
      5 days ago

      DeepSeek the self-hosted model is pretty decent even as distilled down to 8b, but I always ensure i get an abliterated version to remove all the Chinese censorship (and also built-in OpenAI censorship given the actual history of how the model was actually developed).

      • MajinBlayze@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        ·
        5 days ago

        To be clear, that only removes (or attempts to remove) refusals; it doesn’t add in training data that it doesn’t have. Ask it about tiennemen square, for example.

        • BaroqueInMind@lemmy.one
          link
          fedilink
          English
          arrow-up
          6
          ·
          5 days ago

          The abliterated model of DeepSeek can fully discuss Tiananmen Square. I’ve tried to even use the 4chan copy paste that allegedly gets Chinese chat users session dropped and the prompts work fine

              • pezhore@infosec.pub
                link
                fedilink
                English
                arrow-up
                1
                ·
                5 days ago

                I’m doing that with docker compose in my homelab, it’s pretty neat!

                services:
                  ollama:
                    volumes:
                      - /etc/ollama-docker/ollama:/root/.ollama
                    container_name: ollama
                    pull_policy: always
                    tty: true
                    restart: unless-stopped
                    image: ollama/ollama
                    ports:
                      - 11434:11434
                    deploy:
                      resources:
                        reservations:
                          devices:
                            - driver: nvidia
                              device_ids: ['0']
                              capabilities:
                                - gpu
                
                  open-webui:
                    build:
                      context: .
                      args:
                        OLLAMA_BASE_URL: '/ollama'
                      dockerfile: Dockerfile
                    image: ghcr.io/open-webui/open-webui:main
                    container_name: open-webui
                    volumes:
                      - /etc/ollama-docker/open-webui:/app/backend/data
                    depends_on:
                      - ollama
                    ports:
                      - 3000:8080
                    environment:
                      - 'OLLAMA_BASE_URL=http://ollama:11434/'
                      - 'WEBUI_SECRET_KEY='
                    extra_hosts:
                      - host.docker.internal:host-gateway
                    restart: unless-stopped
                
                volumes:
                  ollama: {}
                  open-webui: {}