damjan_savicandClaude Opus 4.5
e1bbe5455d
Vollständige Next.js 15 Portfolio-Website mit: - Blog-System mit 100+ Artikeln - Supabase-Integration - Responsive Design mit Tailwind CSS - TypeScript-Konfiguration - Testing-Setup mit Vitest und Playwright Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
616 lines
18 KiB
Markdown
616 lines
18 KiB
Markdown
# Rate Limiting & Throttling
|
||
|
||
**Meta-Description:** API Rate Limiting implementieren. Token Bucket, Sliding Window und Redis-basierte Rate Limiter für sichere APIs.
|
||
|
||
**Keywords:** Rate Limiting, API Throttling, Token Bucket, Sliding Window, Redis, API Security, DDoS Protection
|
||
|
||
---
|
||
|
||
## Einführung
|
||
|
||
**Rate Limiting** schützt APIs vor Überlastung und Missbrauch. Von **Token Bucket** bis **Sliding Window** – verschiedene Algorithmen haben unterschiedliche Stärken. Dieser Guide zeigt Implementation mit Redis und Best Practices.
|
||
|
||
---
|
||
|
||
## Rate Limiting Overview
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────┐
|
||
│ RATE LIMITING ALGORITHMS │
|
||
├─────────────────────────────────────────────────────────────┤
|
||
│ │
|
||
│ 1. FIXED WINDOW: │
|
||
│ ┌─────────────────────────────────────────────────────┐ │
|
||
│ │ |----Window 1----|----Window 2----| │ │
|
||
│ │ |▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓|░░░░░░░░░░░░░░░| │ │
|
||
│ │ 100 requests/min Reset at boundary │ │
|
||
│ │ ✓ Simple │ │
|
||
│ │ ✗ Burst at window boundary │ │
|
||
│ └─────────────────────────────────────────────────────┘ │
|
||
│ │
|
||
│ 2. SLIDING WINDOW LOG: │
|
||
│ ┌─────────────────────────────────────────────────────┐ │
|
||
│ │ Store timestamps of all requests │ │
|
||
│ │ [t1, t2, t3, ... tn] │ │
|
||
│ │ Count requests in last N seconds │ │
|
||
│ │ ✓ Accurate │ │
|
||
│ │ ✗ Memory intensive │ │
|
||
│ └─────────────────────────────────────────────────────┘ │
|
||
│ │
|
||
│ 3. SLIDING WINDOW COUNTER: │
|
||
│ ┌─────────────────────────────────────────────────────┐ │
|
||
│ │ Previous window: 80 requests │ │
|
||
│ │ Current window: 20 requests │ │
|
||
│ │ Position: 25% into current window │ │
|
||
│ │ Weighted: 80 * 0.75 + 20 = 80 requests │ │
|
||
│ │ ✓ Memory efficient │ │
|
||
│ │ ✓ Smooth rate limiting │ │
|
||
│ └─────────────────────────────────────────────────────┘ │
|
||
│ │
|
||
│ 4. TOKEN BUCKET: │
|
||
│ ┌─────────────────────────────────────────────────────┐ │
|
||
│ │ Bucket capacity: 100 tokens │ │
|
||
│ │ Refill rate: 10 tokens/second │ │
|
||
│ │ Request consumes 1 token │ │
|
||
│ │ ✓ Allows controlled bursts │ │
|
||
│ │ ✓ Smooth average rate │ │
|
||
│ └─────────────────────────────────────────────────────┘ │
|
||
│ │
|
||
│ 5. LEAKY BUCKET: │
|
||
│ ┌─────────────────────────────────────────────────────┐ │
|
||
│ │ Requests enter bucket (queue) │ │
|
||
│ │ Processed at fixed rate (leak) │ │
|
||
│ │ Overflow rejected │ │
|
||
│ │ ✓ Smooth output rate │ │
|
||
│ │ ✗ Doesn't allow bursts │ │
|
||
│ └─────────────────────────────────────────────────────┘ │
|
||
│ │
|
||
└─────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## Redis Rate Limiter
|
||
|
||
```typescript
|
||
// lib/rate-limit/redis.ts
|
||
import { Redis } from 'ioredis';
|
||
|
||
const redis = new Redis(process.env.REDIS_URL!);
|
||
|
||
interface RateLimitConfig {
|
||
identifier: string;
|
||
limit: number;
|
||
window: number; // seconds
|
||
}
|
||
|
||
interface RateLimitResult {
|
||
allowed: boolean;
|
||
remaining: number;
|
||
reset: number;
|
||
retryAfter?: number;
|
||
}
|
||
|
||
// Sliding Window Counter
|
||
export async function slidingWindowRateLimit(
|
||
config: RateLimitConfig
|
||
): Promise<RateLimitResult> {
|
||
const { identifier, limit, window } = config;
|
||
const now = Date.now();
|
||
const windowStart = now - window * 1000;
|
||
const currentWindow = Math.floor(now / (window * 1000));
|
||
const previousWindow = currentWindow - 1;
|
||
|
||
const keys = {
|
||
current: `ratelimit:${identifier}:${currentWindow}`,
|
||
previous: `ratelimit:${identifier}:${previousWindow}`
|
||
};
|
||
|
||
// Get counts from both windows
|
||
const [currentCount, previousCount] = await redis.mget(keys.current, keys.previous);
|
||
const current = parseInt(currentCount || '0');
|
||
const previous = parseInt(previousCount || '0');
|
||
|
||
// Calculate weighted count
|
||
const windowPosition = (now % (window * 1000)) / (window * 1000);
|
||
const weightedCount = Math.floor(previous * (1 - windowPosition) + current);
|
||
|
||
if (weightedCount >= limit) {
|
||
const reset = (currentWindow + 1) * window * 1000;
|
||
return {
|
||
allowed: false,
|
||
remaining: 0,
|
||
reset,
|
||
retryAfter: Math.ceil((reset - now) / 1000)
|
||
};
|
||
}
|
||
|
||
// Increment current window
|
||
await redis
|
||
.multi()
|
||
.incr(keys.current)
|
||
.expire(keys.current, window * 2)
|
||
.exec();
|
||
|
||
return {
|
||
allowed: true,
|
||
remaining: limit - weightedCount - 1,
|
||
reset: (currentWindow + 1) * window * 1000
|
||
};
|
||
}
|
||
|
||
// Token Bucket
|
||
export async function tokenBucketRateLimit(
|
||
identifier: string,
|
||
bucketSize: number,
|
||
refillRate: number // tokens per second
|
||
): Promise<RateLimitResult> {
|
||
const key = `ratelimit:bucket:${identifier}`;
|
||
const now = Date.now();
|
||
|
||
// Lua script for atomic operation
|
||
const luaScript = `
|
||
local key = KEYS[1]
|
||
local bucket_size = tonumber(ARGV[1])
|
||
local refill_rate = tonumber(ARGV[2])
|
||
local now = tonumber(ARGV[3])
|
||
|
||
local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
|
||
local tokens = tonumber(bucket[1]) or bucket_size
|
||
local last_refill = tonumber(bucket[2]) or now
|
||
|
||
-- Calculate tokens to add
|
||
local elapsed = (now - last_refill) / 1000
|
||
local tokens_to_add = elapsed * refill_rate
|
||
tokens = math.min(bucket_size, tokens + tokens_to_add)
|
||
|
||
-- Try to consume a token
|
||
if tokens >= 1 then
|
||
tokens = tokens - 1
|
||
redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
|
||
redis.call('EXPIRE', key, bucket_size / refill_rate * 2)
|
||
return {1, tokens, 0}
|
||
else
|
||
local wait_time = (1 - tokens) / refill_rate
|
||
return {0, tokens, wait_time}
|
||
end
|
||
`;
|
||
|
||
const result = await redis.eval(
|
||
luaScript,
|
||
1,
|
||
key,
|
||
bucketSize,
|
||
refillRate,
|
||
now
|
||
) as [number, number, number];
|
||
|
||
const [allowed, remaining, waitTime] = result;
|
||
|
||
return {
|
||
allowed: allowed === 1,
|
||
remaining: Math.floor(remaining),
|
||
reset: now + Math.ceil((bucketSize - remaining) / refillRate) * 1000,
|
||
retryAfter: waitTime > 0 ? Math.ceil(waitTime) : undefined
|
||
};
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Rate Limit Middleware
|
||
|
||
```typescript
|
||
// lib/rate-limit/middleware.ts
|
||
import { NextResponse } from 'next/server';
|
||
import type { NextRequest } from 'next/server';
|
||
import { slidingWindowRateLimit } from './redis';
|
||
|
||
interface RateLimitTier {
|
||
identifier: (req: NextRequest) => string;
|
||
limit: number;
|
||
window: number;
|
||
}
|
||
|
||
const rateLimitTiers: Record<string, RateLimitTier> = {
|
||
// Public API - by IP
|
||
public: {
|
||
identifier: (req) => req.ip || req.headers.get('x-forwarded-for') || 'unknown',
|
||
limit: 100,
|
||
window: 60 // 100 requests per minute
|
||
},
|
||
// Authenticated API - by user
|
||
authenticated: {
|
||
identifier: (req) => req.headers.get('x-user-id') || 'unknown',
|
||
limit: 1000,
|
||
window: 60 // 1000 requests per minute
|
||
},
|
||
// Premium API - by API key
|
||
premium: {
|
||
identifier: (req) => req.headers.get('x-api-key') || 'unknown',
|
||
limit: 10000,
|
||
window: 60 // 10000 requests per minute
|
||
}
|
||
};
|
||
|
||
export async function rateLimitMiddleware(
|
||
request: NextRequest,
|
||
tier: keyof typeof rateLimitTiers = 'public'
|
||
): Promise<NextResponse | null> {
|
||
const config = rateLimitTiers[tier];
|
||
const identifier = config.identifier(request);
|
||
|
||
const result = await slidingWindowRateLimit({
|
||
identifier: `${tier}:${identifier}`,
|
||
limit: config.limit,
|
||
window: config.window
|
||
});
|
||
|
||
// Set rate limit headers
|
||
const headers = new Headers();
|
||
headers.set('X-RateLimit-Limit', config.limit.toString());
|
||
headers.set('X-RateLimit-Remaining', result.remaining.toString());
|
||
headers.set('X-RateLimit-Reset', result.reset.toString());
|
||
|
||
if (!result.allowed) {
|
||
headers.set('Retry-After', result.retryAfter!.toString());
|
||
|
||
return NextResponse.json(
|
||
{
|
||
error: {
|
||
code: 'RATE_LIMIT_EXCEEDED',
|
||
message: 'Too many requests',
|
||
retryAfter: result.retryAfter
|
||
}
|
||
},
|
||
{
|
||
status: 429,
|
||
headers
|
||
}
|
||
);
|
||
}
|
||
|
||
return null; // Continue to handler
|
||
}
|
||
|
||
// Usage in API route
|
||
// app/api/[...]/route.ts
|
||
export async function GET(request: NextRequest) {
|
||
const rateLimitResponse = await rateLimitMiddleware(request, 'authenticated');
|
||
if (rateLimitResponse) return rateLimitResponse;
|
||
|
||
// Handle request...
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Plan-Based Rate Limits
|
||
|
||
```typescript
|
||
// lib/rate-limit/plans.ts
|
||
import { db } from '@/lib/db';
|
||
import { slidingWindowRateLimit } from './redis';
|
||
|
||
interface PlanLimits {
|
||
requestsPerMinute: number;
|
||
requestsPerDay: number;
|
||
burstSize: number;
|
||
}
|
||
|
||
const planLimits: Record<string, PlanLimits> = {
|
||
free: {
|
||
requestsPerMinute: 60,
|
||
requestsPerDay: 1000,
|
||
burstSize: 10
|
||
},
|
||
pro: {
|
||
requestsPerMinute: 600,
|
||
requestsPerDay: 50000,
|
||
burstSize: 100
|
||
},
|
||
enterprise: {
|
||
requestsPerMinute: 6000,
|
||
requestsPerDay: 1000000,
|
||
burstSize: 1000
|
||
}
|
||
};
|
||
|
||
export async function checkPlanRateLimit(
|
||
userId: string,
|
||
endpoint: string
|
||
): Promise<{ allowed: boolean; error?: string; headers: Record<string, string> }> {
|
||
// Get user's plan
|
||
const user = await db.user.findUnique({
|
||
where: { id: userId },
|
||
select: { plan: true }
|
||
});
|
||
|
||
const plan = user?.plan || 'free';
|
||
const limits = planLimits[plan];
|
||
|
||
// Check per-minute limit
|
||
const minuteResult = await slidingWindowRateLimit({
|
||
identifier: `user:${userId}:minute`,
|
||
limit: limits.requestsPerMinute,
|
||
window: 60
|
||
});
|
||
|
||
// Check daily limit
|
||
const dailyResult = await slidingWindowRateLimit({
|
||
identifier: `user:${userId}:day`,
|
||
limit: limits.requestsPerDay,
|
||
window: 86400
|
||
});
|
||
|
||
const headers = {
|
||
'X-RateLimit-Limit-Minute': limits.requestsPerMinute.toString(),
|
||
'X-RateLimit-Remaining-Minute': minuteResult.remaining.toString(),
|
||
'X-RateLimit-Limit-Day': limits.requestsPerDay.toString(),
|
||
'X-RateLimit-Remaining-Day': dailyResult.remaining.toString()
|
||
};
|
||
|
||
if (!minuteResult.allowed) {
|
||
return {
|
||
allowed: false,
|
||
error: 'Rate limit exceeded (per minute)',
|
||
headers: {
|
||
...headers,
|
||
'Retry-After': minuteResult.retryAfter!.toString()
|
||
}
|
||
};
|
||
}
|
||
|
||
if (!dailyResult.allowed) {
|
||
return {
|
||
allowed: false,
|
||
error: 'Daily rate limit exceeded. Please upgrade your plan.',
|
||
headers: {
|
||
...headers,
|
||
'Retry-After': dailyResult.retryAfter!.toString()
|
||
}
|
||
};
|
||
}
|
||
|
||
return { allowed: true, headers };
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Endpoint-Specific Limits
|
||
|
||
```typescript
|
||
// lib/rate-limit/endpoint.ts
|
||
interface EndpointRateLimit {
|
||
path: string | RegExp;
|
||
method?: string;
|
||
limit: number;
|
||
window: number;
|
||
keyGenerator?: (req: Request) => string;
|
||
}
|
||
|
||
const endpointLimits: EndpointRateLimit[] = [
|
||
// Strict limit for auth endpoints
|
||
{
|
||
path: '/api/auth/login',
|
||
method: 'POST',
|
||
limit: 5,
|
||
window: 300, // 5 attempts per 5 minutes
|
||
keyGenerator: (req) => req.headers.get('x-forwarded-for') || 'unknown'
|
||
},
|
||
// Password reset
|
||
{
|
||
path: '/api/auth/reset-password',
|
||
method: 'POST',
|
||
limit: 3,
|
||
window: 3600 // 3 per hour
|
||
},
|
||
// File uploads
|
||
{
|
||
path: /^\/api\/files\/upload/,
|
||
limit: 10,
|
||
window: 60 // 10 per minute
|
||
},
|
||
// Search (expensive)
|
||
{
|
||
path: '/api/search',
|
||
limit: 30,
|
||
window: 60
|
||
},
|
||
// AI/LLM endpoints
|
||
{
|
||
path: /^\/api\/ai\//,
|
||
limit: 10,
|
||
window: 60
|
||
}
|
||
];
|
||
|
||
export async function getEndpointRateLimit(
|
||
path: string,
|
||
method: string
|
||
): Promise<EndpointRateLimit | null> {
|
||
return endpointLimits.find(limit => {
|
||
const pathMatch = typeof limit.path === 'string'
|
||
? path === limit.path
|
||
: limit.path.test(path);
|
||
|
||
const methodMatch = !limit.method || limit.method === method;
|
||
|
||
return pathMatch && methodMatch;
|
||
}) || null;
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Distributed Rate Limiting
|
||
|
||
```typescript
|
||
// lib/rate-limit/distributed.ts
|
||
import { Redis } from 'ioredis';
|
||
|
||
// Cluster-aware rate limiting
|
||
export class DistributedRateLimiter {
|
||
private redis: Redis;
|
||
private localCounter: Map<string, number> = new Map();
|
||
private syncInterval: number = 1000; // 1 second
|
||
|
||
constructor(redisUrl: string) {
|
||
this.redis = new Redis(redisUrl);
|
||
this.startSync();
|
||
}
|
||
|
||
private startSync() {
|
||
setInterval(async () => {
|
||
// Sync local counters to Redis
|
||
const pipeline = this.redis.pipeline();
|
||
|
||
this.localCounter.forEach((count, key) => {
|
||
if (count > 0) {
|
||
pipeline.incrby(key, count);
|
||
}
|
||
});
|
||
|
||
await pipeline.exec();
|
||
this.localCounter.clear();
|
||
}, this.syncInterval);
|
||
}
|
||
|
||
async checkLimit(
|
||
key: string,
|
||
limit: number,
|
||
window: number
|
||
): Promise<boolean> {
|
||
// Quick local check (approximate)
|
||
const localCount = this.localCounter.get(key) || 0;
|
||
|
||
// Get current count from Redis
|
||
const redisCount = parseInt(await this.redis.get(key) || '0');
|
||
const totalCount = redisCount + localCount;
|
||
|
||
if (totalCount >= limit) {
|
||
return false;
|
||
}
|
||
|
||
// Increment locally
|
||
this.localCounter.set(key, localCount + 1);
|
||
|
||
// Set expiry if new key
|
||
if (redisCount === 0) {
|
||
await this.redis.setex(key, window, '0');
|
||
}
|
||
|
||
return true;
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Rate Limit Dashboard
|
||
|
||
```tsx
|
||
// components/RateLimitStatus.tsx
|
||
'use client';
|
||
|
||
import { useState, useEffect } from 'react';
|
||
|
||
interface RateLimitStatus {
|
||
endpoint: string;
|
||
limit: number;
|
||
used: number;
|
||
remaining: number;
|
||
resetsAt: Date;
|
||
}
|
||
|
||
export function RateLimitStatus({ apiKey }: { apiKey: string }) {
|
||
const [status, setStatus] = useState<RateLimitStatus[]>([]);
|
||
|
||
useEffect(() => {
|
||
fetch(`/api/rate-limits/status?apiKey=${apiKey}`)
|
||
.then(res => res.json())
|
||
.then(setStatus);
|
||
}, [apiKey]);
|
||
|
||
return (
|
||
<div className="bg-white rounded-lg shadow p-6">
|
||
<h2 className="text-lg font-semibold mb-4">Rate Limit Status</h2>
|
||
|
||
<div className="space-y-4">
|
||
{status.map((item) => (
|
||
<div key={item.endpoint} className="border-b pb-4">
|
||
<div className="flex justify-between mb-2">
|
||
<span className="font-medium">{item.endpoint}</span>
|
||
<span className="text-gray-500">
|
||
{item.remaining} / {item.limit} remaining
|
||
</span>
|
||
</div>
|
||
|
||
<div className="w-full bg-gray-200 rounded-full h-2">
|
||
<div
|
||
className={`h-2 rounded-full ${
|
||
item.remaining < item.limit * 0.1
|
||
? 'bg-red-500'
|
||
: item.remaining < item.limit * 0.3
|
||
? 'bg-yellow-500'
|
||
: 'bg-green-500'
|
||
}`}
|
||
style={{ width: `${(item.remaining / item.limit) * 100}%` }}
|
||
/>
|
||
</div>
|
||
|
||
<p className="text-sm text-gray-500 mt-1">
|
||
Resets: {new Date(item.resetsAt).toLocaleTimeString()}
|
||
</p>
|
||
</div>
|
||
))}
|
||
</div>
|
||
</div>
|
||
);
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Best Practices
|
||
|
||
| Aspect | Recommendation |
|
||
|--------|----------------|
|
||
| **Algorithm** | Sliding window for accuracy |
|
||
| **Storage** | Redis for distributed systems |
|
||
| **Headers** | Always return limit headers |
|
||
| **Granularity** | Per-user + per-IP |
|
||
| **Grace Period** | Soft limits before hard block |
|
||
| **Documentation** | Clear rate limit docs |
|
||
|
||
---
|
||
|
||
## Fazit
|
||
|
||
Rate Limiting erfordert:
|
||
|
||
1. **Algorithmus**: Sliding Window oder Token Bucket
|
||
2. **Skalierung**: Redis für verteilte Systeme
|
||
3. **Granularität**: User, IP, Endpoint-spezifisch
|
||
4. **Transparenz**: Headers und Dokumentation
|
||
|
||
Rate Limiting schützt APIs und verbessert Fairness.
|
||
|
||
---
|
||
|
||
## Bildprompts
|
||
|
||
1. "Rate limiting algorithm comparison diagram, token bucket and sliding window"
|
||
2. "API rate limit dashboard, usage gauges and quotas"
|
||
3. "Distributed rate limiting architecture, Redis cluster"
|
||
|
||
---
|
||
|
||
## Quellen
|
||
|
||
- [Stripe Rate Limiting](https://stripe.com/blog/rate-limiters)
|
||
- [Cloudflare Rate Limiting](https://blog.cloudflare.com/counting-things-a-lot-of-different-things/)
|
||
- [Redis Rate Limiting Patterns](https://redis.io/commands/incr#pattern-rate-limiter)
|
||
- [Kong Rate Limiting](https://docs.konghq.com/hub/kong-inc/rate-limiting/)
|