thisago's blog

Isolating LLM Agents

Table of Contents

SaaS LLMs might be a helpful Big Brother. After updating my system reinstalling things from scratch I had the opportunity to re-think some bad habits.

I initially was running the LLM agentic application ordinarily, giving all privileges I have to the LLM inference servers. Including all my sensitive files, environment variables (which some contain secrets) and even sudo, as Qubes's AppVMs are password-less sudo by design.1

Running in a Container

To solve close this large window I used Guix. guix shell can make a --pure --container including only the packages you specify. It's not safe as virtualization but isolates filesystem and reduces shell arsenal with ease, really worth for a start.

The usage is very simple, I mainly use OpenCode so it was basically the needed packages for npm, coreutils for basic Unix-like flow, development SDK and exposed the agent application needed dirs including CWD.

With this bare minimum you can run OpenCode: 

mkdir -p /tmp/opencodeTest
cd /tmp/opencodeTest

guix shell --pure --container --emulate-fhs \
  --network \  # Unfortunately no whitelisting via pure Guix
  --no-cwd \  # Fakes paths
  # Read-write shared paths \
  --share=/home/$USER/.config/opencode/ \
  --share=/home/$USER/.local/share/opencode/ \
  --share=/home/$USER/.local/state/opencode/ \
  --share=/home/$USER/.cache/opencode/ \
  --share=/tmp/opencode/ \  # OpenCode uses this. Sharing makes it persist across sessions
  --share=/home/$USER/.local/share/npm/ \  # Persist OpenCode installed. Everything else outside --share is ephemeral
  --share=$PWD=$HOME/repo \  # Let the agent access the repo, mount in containers's home
  # Read-only paths \
  --expose=/home/$USER/.npmrc \  # In my case is needed to tell npm install in user dir
  --expose=/home/$USER/.config/git/ \
  --expose=/home/$USER/Documents/repos/textsForLlms/build/ \  # This is my prompts that OpenCode config uses, you can see it in codeberg: https://codeberg.org/thisago/textsForLlms
  coreutils git curl ripgrep node bash libgccjit -- \
  sh -c 'npm i -g opencode-linux-x64 && cd repo && ~/.local/share/npm/lib/node_modules/opencode-linux-x64/bin/opencode'
# Before run, please remove the comments :)

# If you never installed OpenCode user-wide, you might need to create all the shared dirs
mkdir -p \
  /home/$USER/.config/opencode/ \
  /home/$USER/.local/share/opencode/ \
  /home/$USER/.local/state/opencode/ \
  /home/$USER/.cache/opencode/ \
  /tmp/opencode/ \
  /home/$USER/.local/share/npm/

Ending

You can achieve this same isolation with Docker as well. It's easy as a couple of flags, but as it's verbose I made it as a Bash script for ease. For safety it only shares your current directory if it's a Git repository. You can check it out: https://codeberg.org/thisago/smallBashScripts/src/branch/master/safeopencode

Footnotes:

1

See docs in GitHub for more details: https://github.com/QubesOS/qubes-doc/blob/2b6397faf087730ccb9874cbce80c1a221b97f0c/user/security-in-qubes/vm-sudo.rst#L10-L54