payload ai

Payload AI Plugin

Upgrading to 4.0.0? Read the migration guide. Reference images and extracted prompt attachments now require an explicit storage resolver.

Transform content creation with intelligent automation — your models, your way

🚀 What is this?

The Payload AI Plugin is your secret weapon for turbocharged content creation. It seamlessly integrates cutting-edge AI capabilities directly into Payload CMS, turning your content workflow from tedious to tremendous.

🎥 See It in Action

• Quick Demo - Watch the magic happen
• Extended Demo - Deep dive into all features
• Customization Guide - Make it your own

⚠️ Beta Notice

This plugin is actively evolving. We're constantly shipping improvements and new features. Tested with Payload v3.88.0.

Quick Start Tip: Try it out with Payload's website template for the smoothest experience.

✨ Features

📝 Text & RichText Fields

Content Generation Magic:

• ✅ Compose - Generate content from scratch
• ✅ Proofread - Polish your prose (Beta)
• ✅ Translate - Break language barriers
• ✅ Rephrase - Find better ways to say it (Beta)
• 🔜 Expand - Elaborate on ideas
• 🔜 Summarize - Distill the essence
• 🔜 Simplify - Make complex things clear

🎨 Upload Fields

• 🎙️ Voice Generation - Powered by ElevenLabs & OpenAI
• 🖼️ Image Generation - Powered by OpenAI (DALL-E & GPT-Image-1)

🔧 Power User Features

• 🔌 Bring Your Own Model - Not limited to our defaults
• 🎛️ Field-Level Prompts - Customize AI behavior per field
• 🔐 Access Control - Lock down who can use AI features
• 🧠 Prompt Editor - Fine-tune AI instructions
• 🌍 i18n Support - Works with your multilingual setup
• 🎨 Custom Components - Extend with your own UI

🔜 Coming Soon

• 📊 Document Analyzer
• ✅ Fact Checking
• 🔄 Automated Workflows
• 💡 Editor Suggestions
• 💬 AI Chat Assistant

📦 Installation

pnpm add @ai-stack/payloadcms

That's it! Now let's configure it.

🛠️ Quick Setup

Step 1: Configure the Plugin

Add to src/payload.config.ts:

import { payloadAiPlugin } from '@ai-stack/payloadcms'

export default buildConfig({
  plugins: [
    payloadAiPlugin({
      collections: {
        [Posts.slug]: true,
      },
      debugging: false,
    }),
  ],
  // ... rest of your config
})

Step 2: Enable AI in Your Fields

Add to your RichText fields (e.g., src/collections/Posts/index.ts):

import { PayloadAiPluginLexicalEditorFeature } from '@ai-stack/payloadcms'

fields: [
  {
    name: 'content',
    type: 'richText',
    editor: lexicalEditor({
      features: ({ rootFeatures }) => {
        return [
          HeadingFeature({ enabledHeadingSizes: ['h1', 'h2', 'h3', 'h4'] }),
          // Add this line:
          PayloadAiPluginLexicalEditorFeature(),
        ]
      },
    }),
  },
]

Step 3: Add Your API Keys

Payload AI can read provider credentials in two ways:

1. Environment variables.
2. Plugin config, useful when credentials are resolved before Payload starts, such as per-instance multi-tenant setups, external secret managers, or tests.

Option A: Environment Variables

Create a .env file in your project root. Add the keys for the providers you want to use:

# Text Generation - Choose your provider(s)
OPENAI_API_KEY=your-openai-api-key           # OpenAI models (GPT-4, etc.)
ANTHROPIC_API_KEY=your-anthropic-api-key     # Claude models
GOOGLE_GENERATIVE_AI_API_KEY=your-google-key # Gemini models
MINIMAX_API_KEY=your-minimax-api-key         # MiniMax M3 / M2.7 models

# Image Generation - Choose your provider(s)
OPENAI_API_KEY=your-openai-api-key           # DALL-E (uses same key as above)
# OPENAI_ORG_ID=your-org-id                  # Required only for GPT-Image-1 model
GOOGLE_GENERATIVE_AI_API_KEY=your-google-key # Imagen (uses same key as above)

# Audio/Voice Generation - Choose your provider(s)
ELEVENLABS_API_KEY=your-elevenlabs-api-key   # ElevenLabs voices
OPENAI_API_KEY=your-openai-api-key           # OpenAI TTS (uses same key as above)

# Optional: Use custom OpenAI-compatible endpoint
# OPENAI_BASE_URL=https://api.openai.com/v1

You only need the keys for the providers you plan to use. Mix and match based on your preferences!

Option B: Plugin Config

If you prefer to resolve secrets yourself at startup, pass them through providers. Config values take priority over environment variables, and omitted values still fall back to .env.

payloadAiPlugin({
  collections: {
    [Posts.slug]: true,
  },
  providers: {
    openai: {
      apiKey: process.env.PAYLOAD_AI_OPENAI_API_KEY,
      baseURL: process.env.PAYLOAD_AI_OPENAI_BASE_URL,
      orgId: process.env.PAYLOAD_AI_OPENAI_ORG_ID,
    },
  },
})

Supported provider keys: openai, anthropic, google, elevenLabs, and minimax.

Important: Restart your server after updating .env or plugin settings!

You may also need to regenerate the import map:

payload generate:importmap

⚙️ Advanced Configuration

resolveReferenceImage: async ({ source, signal, maxBytes }) => {
  if (source.kind !== 'media') {
    throw new Error('Only selected media can be used as reference images.')
  }
  // Application-owned helper: map this authorized ID to your fixed bucket/object key.
  // Enforce maxBytes during loading and honor signal to cancel outstanding work.
  return readReferenceBlobFromStorage({
    collection: source.collection,
    id: source.document.id,
    signal,
    maxBytes,
  })
},

import { payloadAiPlugin } from '@ai-stack/payloadcms'

export default buildConfig({
  plugins: [
    payloadAiPlugin({
      collections: {
        [Posts.slug]: true,
      },

      // Enable AI for globals too
      globals: {
        [Home.slug]: true,
      },

      // Development helpers
      debugging: false,
      disableSponsorMessage: false,
      generatePromptOnInit: process.env.NODE_ENV !== 'production',

      // Specify media collection for GPT-Image-1
      uploadCollectionSlug: 'media',

      // Lock down AI features
      access: {
        generate: ({ req }) => req.user?.role === 'admin',
        settings: ({ req }) => req.user?.role === 'admin',
      },

      // Customize language options
      options: {
        enabledLanguages: ['en-US', 'zh-SG', 'zh-CN', 'en'],
      },

      // Reference additional fields in prompts
      promptFields: [
        {
          name: 'url',
          collections: ['images'],
        },
        {
          name: 'markdown',
          async getter(doc, { collection }) {
            return docToMarkdown(collection, doc)
          },
        },
      ],

      // Control initial prompt generation
      seedPrompts: ({ path }) => {
        if (path.endsWith('.meta.description')) {
          return {
            data: {
              prompt: 'Generate SEO-friendly meta description: {{markdown}}',
            },
          }
        }
        if (path.endsWith('.slug')) return false // Disable for slugs
        return undefined // Use defaults
      },

      // Custom media upload (useful for multi-tenant)
      mediaUpload: async (result, { request, collection }) => {
        return request.payload.create({
          collection,
          data: result.data,
          file: result.file,
          overrideAccess: false,
          req: request,
        })
      },
    }),
  ],
})

@ai-stack/payloadcms/fields#ComposeField

📚 Documentation

Need more details? Check out the Complete Setup Guide for:

• Custom model configuration
• Advanced prompt engineering
• Field-specific customization
• Troubleshooting tips

🤝 Support This Project

Built with ❤️ in my free time. If this plugin saves you hours of work, consider fueling future development!

Every coffee keeps the AI models running and new features shipping. Thank you! 🙏

👥 Contributing

We love contributors! Whether you're fixing typos, suggesting features, or building new capabilities, all contributions are welcome.

Ways to Contribute

• 🐛 Report bugs
• 💡 Suggest features
• 📖 Improve documentation
• 🔧 Submit pull requests

Join the conversation on Payload's Discord and let's build something amazing together!

Local Development

Want to hack on the plugin? Here's how:

Prerequisites

• Node.js (version in .nvmrc)
• pnpm
• Database connection (Postgres or MongoDB)
• Optional: AI provider API keys

Setup

# 1. Install dependencies
pnpm install

# 2. Set up environment
cp dev/.env.example dev/.env
# Edit dev/.env with your DATABASE_URI, PAYLOAD_SECRET, and API keys

# 3. Start development server
pnpm dev
# Admin UI available at http://localhost:3000

# 4. Generate types/importmap if needed
pnpm generate:importmap
pnpm generate:types

Development Workflow

• Plugin source: src/
• Test app: dev/
• Edit files in src/ and refresh to see changes

Testing & Quality

pnpm test                    # Run all tests
pnpm lint                    # ESLint
pnpm prettier --write .      # Format code
pnpm build                   # Build plugin

Test in Another Project

pnpm pack
# In your other project:
pnpm add /path/to/ai-plugin-*.tgz

Project Structure

├── src/              # Plugin source code
├── dev/              # Test Payload app
│   ├── int.spec.ts   # Integration tests
│   └── e2e.spec.ts   # E2E tests
└── README.md         # You are here!

Made with ❤️ and ☕ by the community

Star on GitHub • Join Discord • Read the Guide