


Vincent is the founder and director of Rubix Studios, with over 20 years of experience in branding, marketing, film, photography, and web development. He is a certified partner with industry leaders including Google, Microsoft, AWS, and HubSpot. Vincent also serves as a member of the Maribyrnong City Council Business and Innovation Board and is undertaking an Executive MBA at RMIT University.
This guide outlines a structured approach to implementing a custom OAuth authentication strategy in Payload CMS, using Google as a reference provider. The intent is to explain how Payload expects authentication data to be introduced into its system and how external OAuth providers can be integrated without bypassing Payload’s internal controls.
Rather than positioning OAuth as a standalone integration, this guide treats it as an identity handoff mechanism. Payload remains responsible for user resolution, session issuance, and permission enforcement, while OAuth providers act only as external identity sources.
This document has not been scoped to create users, this can easily be achieved within the auth strategy after the initial payload.find().
Payload does not initiate OAuth flows or communicate directly with external identity providers. Its authentication model assumes that any external identity has already been validated and is presented to Payload in a defined format.
To support this model, OAuth must be implemented as a controlled bridge around Payload’s auth() pipeline. This bridge is responsible for managing cryptographic state, handling redirects, validating provider responses, and translating OAuth data into a form Payload can process.
This separation is intentional. It limits the blast radius of third-party integrations and ensures that Payload remains the single authority for authentication and session management.
A complete OAuth implementation in Payload is composed of four required layers. Each layer performs a specific function and depends on the correct execution of the previous layer.
Payload will not authenticate a user unless all four layers exist and are aligned. When any layer is missing or misconfigured, failures typically present as silent redirects or incomplete sessions rather than explicit errors.
Before implementing Google OAuth, a Google Cloud project must be created and configured with OAuth credentials and authorised redirect URIs.
Create OAuth client

Use the Google Cloud Console to create an OAuth client for a server-side web application.

Once the client is created, Google issues the credentials required to complete the OAuth flow.
GOOGLE_BUSINESS_CLIENT_ID=Client ID
GOOGLE_BUSINESS_CLIENT_SECRET=Client Secret
NEXT_PUBLIC_SERVER_HOSTNAME=localhost or 'yourdomain.com'OAuth flows that use PKCE (Proof Key for Code Exchange) rely on a pair of cryptographically related values to protect against authorization code interception.
Both values must be generated server-side. The verifier must be persisted temporarily so it can be recovered during the callback phase. At no point should these values be exposed to the client.
To enforce this boundary, PKCE logic should be isolated into a small utility module responsible only for generation and encoding. This keeps cryptographic concerns separate from routing and provider logic and simplifies audit and reuse.
lib/auth/pke.ts
import crypto from 'crypto'
export enum CodeChallengeMethod {
Plain = 'plain',
S256 = 'S256',
}
export function generateCodeChallenge(codeVerifier: string) {
const hash = crypto.createHash('sha256').update(codeVerifier).digest()
return base64URLEncode(hash)
}
export function generateCodeVerifier() {
return base64URLEncode(crypto.randomBytes(32))
}
function base64URLEncode(buffer: Buffer) {
return buffer.toString('base64').replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '')
}The authorisation endpoint initiates the OAuth flow. It does not authenticate users and does not interact with Payload directly.
Its role is to prepare and persist the temporary state required for OAuth, then redirect the user to the provider. Specifically, it is responsible for:
For Google, this endpoint is exposed at /api/users/auth/google.
Cookies are short-lived, scoped conservatively, and cleared after use. They exist solely to link the authorization request to the callback response and should never persist beyond that window.
Endpoint: /api/users/auth/google
import { google } from 'googleapis'
import { type Endpoint, type PayloadRequest } from 'payload'
import { CodeChallengeMethod, generateCodeChallenge, generateCodeVerifier } from '@/lib/auth/pke'
import { getServerSideURL } from '@/utils/getURL'
const isProduction = process.env.NODE_ENV === 'production'
const clientId = process.env.GOOGLE_BUSINESS_CLIENT_ID!
const clientSecret = process.env.GOOGLE_BUSINESS_CLIENT_SECRET!
const hostname = process.env.NEXT_PUBLIC_SERVER_HOSTNAME!
export const googleAuth: Endpoint = {
handler: async (req: PayloadRequest): Promise<Response> => {
const url = new URL(req.url ?? '')
const consentFlag = url.searchParams.get('force_consent') === 'true'
const oauth2Client = new google.auth.OAuth2({
clientId,
clientSecret,
redirectUri: `${getServerSideURL()}/api/users/auth/google/callback`,
})
try {
const codeVerifier = generateCodeVerifier()
const codeChallenge = generateCodeChallenge(codeVerifier)
const authUrl = oauth2Client.generateAuthUrl({
access_type: 'offline',
code_challenge: codeChallenge,
code_challenge_method: CodeChallengeMethod.S256,
prompt: consentFlag ? 'consent' : 'none',
scope: [
'https://www.googleapis.com/auth/userinfo.email',
'https://www.googleapis.com/auth/userinfo.profile',
],
})
const domain =
isProduction && hostname ? hostname.replace(/^https?:\/\//, '').split(':')[0] : undefined
const cookieParts = [
`codeVerifier=${codeVerifier}`,
`strategies=google`,
'Path=/',
'HttpOnly',
'Max-Age=300',
'SameSite=Lax',
isProduction ? 'Secure' : '',
domain ? `Domain=${domain}` : '',
].filter(Boolean)
const cookieHeader = cookieParts.join('; ')
if (consentFlag) {
return new Response(null, {
headers: {
Location: authUrl,
'Set-Cookie': cookieHeader,
},
status: 302,
})
}
return new Response(JSON.stringify({ url: authUrl }), {
headers: {
'Content-Type': 'application/json',
'Set-Cookie': cookieHeader,
},
status: 200,
})
} catch (error) {
console.error('Authentication error:', error)
return new Response(JSON.stringify({ error: 'Failed to authenticate' }), {
headers: { 'Content-Type': 'application/json' },
status: 500,
})
}
},
method: 'get',
path: '/auth/google',
}The callback endpoint is where control returns to the application after the OAuth provider completes authentication.
This endpoint validates the provider response and hands off identity resolution to Payload. Its responsibilities include:
payload.auth() with OAuth data supplied via request headersPayload does not accept OAuth data through query parameters or request bodies. All OAuth context must be provided through headers so it can be processed by the registered authentication strategy.
If Payload successfully resolves a user, the callback endpoint proceeds to issue a signed session token, generate Payload authentication cookies, and redirect the user into the authenticated admin interface. If any step fails, the user is redirected with an explicit error state and no residual authentication data.
Endpoint: /api/users/auth/google/callback
import configPromise from '@payload-config'
import crypto from 'crypto'
import {
type Endpoint,
generatePayloadCookie,
getPayload,
jwtSign,
parseCookies,
type PayloadRequest,
type SanitizedPermissions,
type TypedUser,
} from 'payload'
import { getServerSideURL } from '@/utils/getURL'
const isProduction = process.env.NODE_ENV === 'production'
interface AuthResult {
permissions: SanitizedPermissions
responseHeaders?: Headers
user:
| null
| (TypedUser & {
_sid?: string
_strategy?: string
collection: 'users'
})
}
export const googleCallback: Endpoint = {
handler: async (req: PayloadRequest): Promise<Response> => {
try {
const payload = await getPayload({ config: configPromise })
const url = new URL(req.url ?? getServerSideURL())
const code = url.searchParams.get('code')
const error = url.searchParams.get('error')
const clearCookie = [
'codeVerifier=',
'strategies=',
'Path=/',
'HttpOnly',
'Max-Age=0',
'SameSite=Lax',
isProduction ? 'Secure' : '',
]
.filter(Boolean)
.join('; ')
const cookie = parseCookies(req.headers)
const codeVerifier = cookie.get('codeVerifier')
const strategies = cookie.get('strategies')
if (
error === 'interaction_required' ||
error === 'login_required' ||
error === 'consent_required'
) {
return new Response(null, {
headers: {
Location: `${getServerSideURL()}/api/users/auth/google?force_consent=true`,
'Set-Cookie': clearCookie,
},
status: 302,
})
}
const errorRedirect = (reason: string) =>
new Response(null, {
headers: {
Location: `${getServerSideURL()}/admin/login?error=${reason}`,
'Set-Cookie': clearCookie,
},
status: 302,
})
if (!code || !codeVerifier) {
console.error('Missing OAuth parameters', { code, codeVerifier })
return errorRedirect('missing_parameters')
}
const authResult = await payload.auth({
headers: new Headers({
'x-auth-strategy': strategies || '',
'x-oauth-code': code,
'x-oauth-verifier': codeVerifier,
}),
})
const { user } = authResult as AuthResult
if (!user) {
console.error('Authentication failed: No user returned')
return new Response(null, {
headers: {
Location: `${getServerSideURL()}/admin/login?error=auth_failed`,
'Set-Cookie': clearCookie,
},
status: 302,
})
}
const collection = payload.collections['users']
const authConfig = collection.config.auth
const secret = crypto
.createHash('sha256')
.update(payload.config.secret)
.digest('hex')
.slice(0, 32)
const { token } = await jwtSign({
fieldsToSign: {
_strategy: user._strategy ?? undefined,
collection: 'users',
email: user.email,
id: user.id,
sid: user._sid ?? undefined,
},
secret,
tokenExpiration: authConfig.tokenExpiration,
})
const cookies = generatePayloadCookie({
collectionAuthConfig: authConfig,
cookiePrefix: payload.config.cookiePrefix,
token: token!,
})
return new Response(null, {
headers: {
Location: `${getServerSideURL()}/admin`,
'Set-Cookie': cookies,
},
status: 302,
})
} catch (err) {
console.error('Google OAuth callback error:', err)
const clearCookie = [
'codeVerifier=',
'strategies=',
'Path=/',
'HttpOnly',
'Max-Age=0',
'SameSite=Lax',
isProduction ? 'Secure' : '',
]
.filter(Boolean)
.join('; ')
return new Response(null, {
headers: {
Location: `${getServerSideURL()}/admin/login?error=oauth_failed`,
'Set-Cookie': clearCookie,
},
status: 302,
})
}
},
method: 'get',
path: '/auth/google/callback',
}The authentication strategy is where external OAuth identity is converted into a Payload user.
This strategy is registered in the Payload configuration and is invoked internally when payload.auth() is called with a matching strategy identifier. It is responsible for:
{ user } object to PayloadPayload will not issue a session unless this strategy returns a valid user. All provider communication and token handling must remain encapsulated within this layer to avoid leaking OAuth concerns into the wider application.
import crypto from 'crypto'
import { google } from 'googleapis'
import { type AuthStrategy, type AuthStrategyResult } from 'payload'
import { type User } from '@/payload/payload-types'
import { getServerSideURL } from '@/utils/getURL'
import { mergeAuth } from '@/utils/mergeAuth'
interface SessionUser extends User {
_sid?: string
_strategy?: string
collection: 'users'
}
const clientId = process.env.GOOGLE_BUSINESS_CLIENT_ID!
const clientSecret = process.env.GOOGLE_BUSINESS_CLIENT_SECRET!
export const googleStrategy: AuthStrategy = {
async authenticate({ headers, payload }): Promise<AuthStrategyResult> {
const code = headers.get('x-oauth-code')
const codeVerifier = headers.get('x-oauth-verifier')
const strategy = headers.get('x-auth-strategy')
if (strategy !== 'google') return { user: null }
if (!code || !codeVerifier) return { user: null }
const oauth2Client = new google.auth.OAuth2({
clientId,
clientSecret,
redirectUri: `${getServerSideURL()}/api/users/auth/google/callback`,
})
const { tokens } = await oauth2Client.getToken({ code, codeVerifier })
headers.delete('x-auth-strategy')
headers.delete('x-oauth-code')
headers.delete('x-oauth-verifier')
if (!tokens.access_token) return { user: null }
oauth2Client.setCredentials(tokens)
const oauth2 = google.oauth2({ auth: oauth2Client, version: 'v2' })
const { data: profile } = await oauth2.userinfo.get()
const email = profile.email?.toLowerCase()
if (!email) return { user: null }
const user = (
await payload.find({
collection: 'users',
limit: 1,
pagination: false,
showHiddenFields: true,
where: {
email: {
equals: email,
},
},
})
).docs[0] as SessionUser | undefined
if (!user) {
return { user: null }
}
const collection = payload.collections['users']
const authConfig = collection.config.auth
let sid: string | undefined
const now = new Date()
const tokenExpInMs = (authConfig.tokenExpiration || 7200) * 1000
const expiresAt = new Date(now.getTime() + tokenExpInMs)
if (authConfig.useSessions) {
sid = crypto.randomUUID()
const session = {
createdAt: now.toISOString(),
expiresAt: expiresAt.toISOString(),
id: sid,
}
user.sessions = Array.isArray(user.sessions)
? [...removeExpiredSessions(user.sessions), session]
: [session]
}
const strategies = user.externalId?.authStrategies ?? []
const existingIndex = strategies.findIndex((s) => s.authProvider === 'google')
const existing = existingIndex >= 0 ? strategies[existingIndex] : {}
const googleAuth = mergeAuth(existing, {
accessToken: tokens.access_token,
authProvider: 'google',
idToken: tokens.id_token,
providerUserId: profile.id,
refreshToken: tokens.refresh_token,
tokenExpiry: tokens.expiry_date ? new Date(tokens.expiry_date).toISOString() : undefined,
tokenType: tokens.token_type ?? 'Bearer',
})
if (existingIndex >= 0) {
strategies[existingIndex] = { ...strategies[existingIndex], ...googleAuth }
} else {
strategies.push(googleAuth)
}
await payload.db.updateOne({
collection: 'users',
data: {
externalId: {
...(user.externalId || {}),
authStrategies: strategies,
},
sessions: user.sessions,
},
id: user.id,
returning: false,
})
const sessionUser: SessionUser = {
...user,
_sid: sid,
_strategy: 'google',
collection: 'users',
}
return { user: sessionUser }
},
name: 'google',
}
export const removeExpiredSessions = (
sessions: {
createdAt?: null | string
expiresAt: string
id: string
}[]
) => {
const now = new Date()
return sessions.filter(({ expiresAt }) => new Date(expiresAt) > now)
}/utils/mergeAuth.ts
export function mergeAuth<T extends object>(existing: T | undefined, updates: Partial<T>): T {
const base = (existing ?? {}) as T
const pick = <K extends keyof T>(key: K): NonNullable<T[K]> | null => {
const updateVal = updates[key]
if (updateVal === undefined || updateVal === '' || updateVal === null) {
return base[key] ?? null
}
return updateVal
}
const result = {} as T
for (const key of Object.keys({ ...base, ...updates }) as (keyof T)[]) {
// @ts-expect-error We intentionally allow null for missing values
result[key] = pick(key)
}
return result
}The client application does not interact with OAuth providers directly.
Its sole responsibility is to initiate the authorisation endpoint and redirect the browser using the URL returned by the server. This ensures that PKCE values, provider credentials, and OAuth state remain entirely server-controlled.
This pattern allows additional providers to be introduced without altering the frontend logic and avoids duplicating authentication behavior across clients.
'use client'
import { useState } from 'react'
import { FcGoogle } from 'react-icons/fc'
import { Button } from '@/components/ui/button'
import { type User } from '@/payload/payload-types'
type Provider = NonNullable<
NonNullable<User['externalId']>['authStrategies']
>[number]['authProvider']
export const AfterLogin = () => {
const [loading, setLoading] = useState(false)
const handleLogin = (provider: Provider) => async () => {
setLoading(true)
try {
const res = await fetch(`/api/users/auth/${provider}`)
if (!res.ok) throw new Error(`Failed to start ${provider} auth flow`)
const data = await res.json()
if (!data.url) {
throw new Error('Missing auth URL')
}
window.location.href = data.url
} catch (err) {
console.error(err)
alert('Error starting Google login')
setLoading(false)
}
}
return (
<div className="twp">
<div className="my-4 flex justify-center">
<Button
disabled={loading}
onClick={handleLogin('google')}
className="flex cursor-pointer items-center gap-2 rounded-4xl border-border p-6 text-base font-semibold text-foreground shadow transition-all duration-300 hover:shadow-md"
size="lg"
variant="outline"
>
<FcGoogle className="size-6" />
{loading ? 'Redirecting…' : 'Sign in with Google'}
</Button>
</div>
</div>
)
}Before any OAuth flow can be initiated, the Payload user collection must be configured to accept external authentication strategies and issue sessions programmatically.
Payload does not infer authentication behaviour at runtime. The collection must explicitly declare that it supports authentication, how tokens are generated, and which external strategies are permitted. Without this configuration, OAuth callbacks may resolve identity but will fail to produce a valid session.
At a minimum, the user collection must enable authentication and expose its API endpoints. Token expiration and cookie behaviour should be defined here to ensure consistency across native and external login methods.
A typical configuration includes:
auth on the collectionThis configuration establishes the contract that all subsequent OAuth layers rely on.
import { type CollectionConfig } from 'payload'
import { googleAuth } from '@/payload/collections/users/api/auth/google/auth'
import { googleCallback } from '@/payload/collections/users/api/auth/google/callback'
import { googleStrategy } from '@/payload/collections/users/strategies/google'
export const Users: CollectionConfig = {
...
slug: 'users',
auth: {
cookies: {
domain: process.env.NEXT_PUBLIC_SERVER_HOSTNAME!,
sameSite: 'Lax',
secure: process.env.NODE_ENV === 'production',
},
loginWithUsername: {
allowEmailLogin: true,
requireEmail: true,
requireUsername: true,
},
maxLoginAttempts: 5,
strategies: [googleStrategy],
tokenExpiration: 60 * 60 * 24 * 7,
useSessions: true,
},
endpoints: [googleAuth, googleCallback],
fields: [
{
tabs: [
{
fields: [
{
fields: [
{
admin: {
hidden: true,
readOnly: true,
},
name: 'providerUserId',
type: 'text',
},
{
admin: {
hidden: true,
readOnly: true,
},
name: 'accessToken',
type: 'text',
},
{
admin: {
hidden: true,
readOnly: true,
},
name: 'refreshToken',
type: 'text',
},
{
admin: {
hidden: true,
readOnly: true,
},
name: 'tokenType',
type: 'text',
},
{
admin: {
hidden: true,
readOnly: true,
},
name: 'idToken',
type: 'text',
},
{
admin: {
hidden: true,
readOnly: true,
},
name: 'tokenExpiry',
type: 'date',
},
{
admin: {
hidden: true,
readOnly: true,
},
name: 'authProvider',
options: [
{
label: 'Google',
value: 'google',
},
],
type: 'select',
},
],
label: 'Strategies',
name: 'authStrategies',
type: 'array',
},
],
label: 'External ID',
name: 'externalId',
},
],
type: 'tabs',
},
],
...
}It is strongly recommended to enable field-level encryption to ensure that access and refresh tokens are stored and transmitted in encrypted form.
A custom OAuth strategy in Payload is a coordinated system rather than a single integration point. Implementing components out of sequence often results in incomplete state handling and difficult-to-diagnose failures.
The recommended implementation order is:
Following this order ensures that cryptographic state, OAuth validation, and Payload session issuance remain consistent throughout the flow.
If this has helped you at all, help us out by leaving us a review: https://g.page/r/CcUcaY_5f4XrEBM/review
If you do need help or have questions feel free to leave a comment.
OAuth authentication in Payload is best approached as a controlled identity handoff rather than a direct provider integration. By isolating responsibilities across clearly defined layers, teams can introduce external identity providers while preserving Payload’s authority over authentication, sessions, and permissions.
This structure scales cleanly to additional providers and supports predictable, auditable authentication behavior across environments.
Vincent is the founder and director of Rubix Studios, with over 20 years of experience in branding, marketing, film, photography, and web development. He is a certified partner with industry leaders including Google, Microsoft, AWS, and HubSpot. Vincent also serves as a member of the Maribyrnong City Council Business and Innovation Board and is undertaking an Executive MBA at RMIT University.