Add blog posts, cleanup unused files, update components
- Add 100 blog posts covering AI, development, and tech topics - Add .env.example for environment configuration - Add accessibility and lighthouse audit scripts - Remove obsolete SEO reports and temporary files - Remove dev-dist build artifacts and backup files - Remove unused portrait images (moved/consolidated elsewhere) - Update contact form and component improvements Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,615 @@
|
||||
# Rate Limiting & Throttling
|
||||
|
||||
**Meta-Description:** API Rate Limiting implementieren. Token Bucket, Sliding Window und Redis-basierte Rate Limiter für sichere APIs.
|
||||
|
||||
**Keywords:** Rate Limiting, API Throttling, Token Bucket, Sliding Window, Redis, API Security, DDoS Protection
|
||||
|
||||
---
|
||||
|
||||
## Einführung
|
||||
|
||||
**Rate Limiting** schützt APIs vor Überlastung und Missbrauch. Von **Token Bucket** bis **Sliding Window** – verschiedene Algorithmen haben unterschiedliche Stärken. Dieser Guide zeigt Implementation mit Redis und Best Practices.
|
||||
|
||||
---
|
||||
|
||||
## Rate Limiting Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ RATE LIMITING ALGORITHMS │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 1. FIXED WINDOW: │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ |----Window 1----|----Window 2----| │ │
|
||||
│ │ |▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓|░░░░░░░░░░░░░░░| │ │
|
||||
│ │ 100 requests/min Reset at boundary │ │
|
||||
│ │ ✓ Simple │ │
|
||||
│ │ ✗ Burst at window boundary │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 2. SLIDING WINDOW LOG: │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Store timestamps of all requests │ │
|
||||
│ │ [t1, t2, t3, ... tn] │ │
|
||||
│ │ Count requests in last N seconds │ │
|
||||
│ │ ✓ Accurate │ │
|
||||
│ │ ✗ Memory intensive │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. SLIDING WINDOW COUNTER: │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Previous window: 80 requests │ │
|
||||
│ │ Current window: 20 requests │ │
|
||||
│ │ Position: 25% into current window │ │
|
||||
│ │ Weighted: 80 * 0.75 + 20 = 80 requests │ │
|
||||
│ │ ✓ Memory efficient │ │
|
||||
│ │ ✓ Smooth rate limiting │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 4. TOKEN BUCKET: │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Bucket capacity: 100 tokens │ │
|
||||
│ │ Refill rate: 10 tokens/second │ │
|
||||
│ │ Request consumes 1 token │ │
|
||||
│ │ ✓ Allows controlled bursts │ │
|
||||
│ │ ✓ Smooth average rate │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 5. LEAKY BUCKET: │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Requests enter bucket (queue) │ │
|
||||
│ │ Processed at fixed rate (leak) │ │
|
||||
│ │ Overflow rejected │ │
|
||||
│ │ ✓ Smooth output rate │ │
|
||||
│ │ ✗ Doesn't allow bursts │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Redis Rate Limiter
|
||||
|
||||
```typescript
|
||||
// lib/rate-limit/redis.ts
|
||||
import { Redis } from 'ioredis';
|
||||
|
||||
const redis = new Redis(process.env.REDIS_URL!);
|
||||
|
||||
interface RateLimitConfig {
|
||||
identifier: string;
|
||||
limit: number;
|
||||
window: number; // seconds
|
||||
}
|
||||
|
||||
interface RateLimitResult {
|
||||
allowed: boolean;
|
||||
remaining: number;
|
||||
reset: number;
|
||||
retryAfter?: number;
|
||||
}
|
||||
|
||||
// Sliding Window Counter
|
||||
export async function slidingWindowRateLimit(
|
||||
config: RateLimitConfig
|
||||
): Promise<RateLimitResult> {
|
||||
const { identifier, limit, window } = config;
|
||||
const now = Date.now();
|
||||
const windowStart = now - window * 1000;
|
||||
const currentWindow = Math.floor(now / (window * 1000));
|
||||
const previousWindow = currentWindow - 1;
|
||||
|
||||
const keys = {
|
||||
current: `ratelimit:${identifier}:${currentWindow}`,
|
||||
previous: `ratelimit:${identifier}:${previousWindow}`
|
||||
};
|
||||
|
||||
// Get counts from both windows
|
||||
const [currentCount, previousCount] = await redis.mget(keys.current, keys.previous);
|
||||
const current = parseInt(currentCount || '0');
|
||||
const previous = parseInt(previousCount || '0');
|
||||
|
||||
// Calculate weighted count
|
||||
const windowPosition = (now % (window * 1000)) / (window * 1000);
|
||||
const weightedCount = Math.floor(previous * (1 - windowPosition) + current);
|
||||
|
||||
if (weightedCount >= limit) {
|
||||
const reset = (currentWindow + 1) * window * 1000;
|
||||
return {
|
||||
allowed: false,
|
||||
remaining: 0,
|
||||
reset,
|
||||
retryAfter: Math.ceil((reset - now) / 1000)
|
||||
};
|
||||
}
|
||||
|
||||
// Increment current window
|
||||
await redis
|
||||
.multi()
|
||||
.incr(keys.current)
|
||||
.expire(keys.current, window * 2)
|
||||
.exec();
|
||||
|
||||
return {
|
||||
allowed: true,
|
||||
remaining: limit - weightedCount - 1,
|
||||
reset: (currentWindow + 1) * window * 1000
|
||||
};
|
||||
}
|
||||
|
||||
// Token Bucket
|
||||
export async function tokenBucketRateLimit(
|
||||
identifier: string,
|
||||
bucketSize: number,
|
||||
refillRate: number // tokens per second
|
||||
): Promise<RateLimitResult> {
|
||||
const key = `ratelimit:bucket:${identifier}`;
|
||||
const now = Date.now();
|
||||
|
||||
// Lua script for atomic operation
|
||||
const luaScript = `
|
||||
local key = KEYS[1]
|
||||
local bucket_size = tonumber(ARGV[1])
|
||||
local refill_rate = tonumber(ARGV[2])
|
||||
local now = tonumber(ARGV[3])
|
||||
|
||||
local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
|
||||
local tokens = tonumber(bucket[1]) or bucket_size
|
||||
local last_refill = tonumber(bucket[2]) or now
|
||||
|
||||
-- Calculate tokens to add
|
||||
local elapsed = (now - last_refill) / 1000
|
||||
local tokens_to_add = elapsed * refill_rate
|
||||
tokens = math.min(bucket_size, tokens + tokens_to_add)
|
||||
|
||||
-- Try to consume a token
|
||||
if tokens >= 1 then
|
||||
tokens = tokens - 1
|
||||
redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
|
||||
redis.call('EXPIRE', key, bucket_size / refill_rate * 2)
|
||||
return {1, tokens, 0}
|
||||
else
|
||||
local wait_time = (1 - tokens) / refill_rate
|
||||
return {0, tokens, wait_time}
|
||||
end
|
||||
`;
|
||||
|
||||
const result = await redis.eval(
|
||||
luaScript,
|
||||
1,
|
||||
key,
|
||||
bucketSize,
|
||||
refillRate,
|
||||
now
|
||||
) as [number, number, number];
|
||||
|
||||
const [allowed, remaining, waitTime] = result;
|
||||
|
||||
return {
|
||||
allowed: allowed === 1,
|
||||
remaining: Math.floor(remaining),
|
||||
reset: now + Math.ceil((bucketSize - remaining) / refillRate) * 1000,
|
||||
retryAfter: waitTime > 0 ? Math.ceil(waitTime) : undefined
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rate Limit Middleware
|
||||
|
||||
```typescript
|
||||
// lib/rate-limit/middleware.ts
|
||||
import { NextResponse } from 'next/server';
|
||||
import type { NextRequest } from 'next/server';
|
||||
import { slidingWindowRateLimit } from './redis';
|
||||
|
||||
interface RateLimitTier {
|
||||
identifier: (req: NextRequest) => string;
|
||||
limit: number;
|
||||
window: number;
|
||||
}
|
||||
|
||||
const rateLimitTiers: Record<string, RateLimitTier> = {
|
||||
// Public API - by IP
|
||||
public: {
|
||||
identifier: (req) => req.ip || req.headers.get('x-forwarded-for') || 'unknown',
|
||||
limit: 100,
|
||||
window: 60 // 100 requests per minute
|
||||
},
|
||||
// Authenticated API - by user
|
||||
authenticated: {
|
||||
identifier: (req) => req.headers.get('x-user-id') || 'unknown',
|
||||
limit: 1000,
|
||||
window: 60 // 1000 requests per minute
|
||||
},
|
||||
// Premium API - by API key
|
||||
premium: {
|
||||
identifier: (req) => req.headers.get('x-api-key') || 'unknown',
|
||||
limit: 10000,
|
||||
window: 60 // 10000 requests per minute
|
||||
}
|
||||
};
|
||||
|
||||
export async function rateLimitMiddleware(
|
||||
request: NextRequest,
|
||||
tier: keyof typeof rateLimitTiers = 'public'
|
||||
): Promise<NextResponse | null> {
|
||||
const config = rateLimitTiers[tier];
|
||||
const identifier = config.identifier(request);
|
||||
|
||||
const result = await slidingWindowRateLimit({
|
||||
identifier: `${tier}:${identifier}`,
|
||||
limit: config.limit,
|
||||
window: config.window
|
||||
});
|
||||
|
||||
// Set rate limit headers
|
||||
const headers = new Headers();
|
||||
headers.set('X-RateLimit-Limit', config.limit.toString());
|
||||
headers.set('X-RateLimit-Remaining', result.remaining.toString());
|
||||
headers.set('X-RateLimit-Reset', result.reset.toString());
|
||||
|
||||
if (!result.allowed) {
|
||||
headers.set('Retry-After', result.retryAfter!.toString());
|
||||
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: {
|
||||
code: 'RATE_LIMIT_EXCEEDED',
|
||||
message: 'Too many requests',
|
||||
retryAfter: result.retryAfter
|
||||
}
|
||||
},
|
||||
{
|
||||
status: 429,
|
||||
headers
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
return null; // Continue to handler
|
||||
}
|
||||
|
||||
// Usage in API route
|
||||
// app/api/[...]/route.ts
|
||||
export async function GET(request: NextRequest) {
|
||||
const rateLimitResponse = await rateLimitMiddleware(request, 'authenticated');
|
||||
if (rateLimitResponse) return rateLimitResponse;
|
||||
|
||||
// Handle request...
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Plan-Based Rate Limits
|
||||
|
||||
```typescript
|
||||
// lib/rate-limit/plans.ts
|
||||
import { db } from '@/lib/db';
|
||||
import { slidingWindowRateLimit } from './redis';
|
||||
|
||||
interface PlanLimits {
|
||||
requestsPerMinute: number;
|
||||
requestsPerDay: number;
|
||||
burstSize: number;
|
||||
}
|
||||
|
||||
const planLimits: Record<string, PlanLimits> = {
|
||||
free: {
|
||||
requestsPerMinute: 60,
|
||||
requestsPerDay: 1000,
|
||||
burstSize: 10
|
||||
},
|
||||
pro: {
|
||||
requestsPerMinute: 600,
|
||||
requestsPerDay: 50000,
|
||||
burstSize: 100
|
||||
},
|
||||
enterprise: {
|
||||
requestsPerMinute: 6000,
|
||||
requestsPerDay: 1000000,
|
||||
burstSize: 1000
|
||||
}
|
||||
};
|
||||
|
||||
export async function checkPlanRateLimit(
|
||||
userId: string,
|
||||
endpoint: string
|
||||
): Promise<{ allowed: boolean; error?: string; headers: Record<string, string> }> {
|
||||
// Get user's plan
|
||||
const user = await db.user.findUnique({
|
||||
where: { id: userId },
|
||||
select: { plan: true }
|
||||
});
|
||||
|
||||
const plan = user?.plan || 'free';
|
||||
const limits = planLimits[plan];
|
||||
|
||||
// Check per-minute limit
|
||||
const minuteResult = await slidingWindowRateLimit({
|
||||
identifier: `user:${userId}:minute`,
|
||||
limit: limits.requestsPerMinute,
|
||||
window: 60
|
||||
});
|
||||
|
||||
// Check daily limit
|
||||
const dailyResult = await slidingWindowRateLimit({
|
||||
identifier: `user:${userId}:day`,
|
||||
limit: limits.requestsPerDay,
|
||||
window: 86400
|
||||
});
|
||||
|
||||
const headers = {
|
||||
'X-RateLimit-Limit-Minute': limits.requestsPerMinute.toString(),
|
||||
'X-RateLimit-Remaining-Minute': minuteResult.remaining.toString(),
|
||||
'X-RateLimit-Limit-Day': limits.requestsPerDay.toString(),
|
||||
'X-RateLimit-Remaining-Day': dailyResult.remaining.toString()
|
||||
};
|
||||
|
||||
if (!minuteResult.allowed) {
|
||||
return {
|
||||
allowed: false,
|
||||
error: 'Rate limit exceeded (per minute)',
|
||||
headers: {
|
||||
...headers,
|
||||
'Retry-After': minuteResult.retryAfter!.toString()
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
if (!dailyResult.allowed) {
|
||||
return {
|
||||
allowed: false,
|
||||
error: 'Daily rate limit exceeded. Please upgrade your plan.',
|
||||
headers: {
|
||||
...headers,
|
||||
'Retry-After': dailyResult.retryAfter!.toString()
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
return { allowed: true, headers };
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Endpoint-Specific Limits
|
||||
|
||||
```typescript
|
||||
// lib/rate-limit/endpoint.ts
|
||||
interface EndpointRateLimit {
|
||||
path: string | RegExp;
|
||||
method?: string;
|
||||
limit: number;
|
||||
window: number;
|
||||
keyGenerator?: (req: Request) => string;
|
||||
}
|
||||
|
||||
const endpointLimits: EndpointRateLimit[] = [
|
||||
// Strict limit for auth endpoints
|
||||
{
|
||||
path: '/api/auth/login',
|
||||
method: 'POST',
|
||||
limit: 5,
|
||||
window: 300, // 5 attempts per 5 minutes
|
||||
keyGenerator: (req) => req.headers.get('x-forwarded-for') || 'unknown'
|
||||
},
|
||||
// Password reset
|
||||
{
|
||||
path: '/api/auth/reset-password',
|
||||
method: 'POST',
|
||||
limit: 3,
|
||||
window: 3600 // 3 per hour
|
||||
},
|
||||
// File uploads
|
||||
{
|
||||
path: /^\/api\/files\/upload/,
|
||||
limit: 10,
|
||||
window: 60 // 10 per minute
|
||||
},
|
||||
// Search (expensive)
|
||||
{
|
||||
path: '/api/search',
|
||||
limit: 30,
|
||||
window: 60
|
||||
},
|
||||
// AI/LLM endpoints
|
||||
{
|
||||
path: /^\/api\/ai\//,
|
||||
limit: 10,
|
||||
window: 60
|
||||
}
|
||||
];
|
||||
|
||||
export async function getEndpointRateLimit(
|
||||
path: string,
|
||||
method: string
|
||||
): Promise<EndpointRateLimit | null> {
|
||||
return endpointLimits.find(limit => {
|
||||
const pathMatch = typeof limit.path === 'string'
|
||||
? path === limit.path
|
||||
: limit.path.test(path);
|
||||
|
||||
const methodMatch = !limit.method || limit.method === method;
|
||||
|
||||
return pathMatch && methodMatch;
|
||||
}) || null;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Distributed Rate Limiting
|
||||
|
||||
```typescript
|
||||
// lib/rate-limit/distributed.ts
|
||||
import { Redis } from 'ioredis';
|
||||
|
||||
// Cluster-aware rate limiting
|
||||
export class DistributedRateLimiter {
|
||||
private redis: Redis;
|
||||
private localCounter: Map<string, number> = new Map();
|
||||
private syncInterval: number = 1000; // 1 second
|
||||
|
||||
constructor(redisUrl: string) {
|
||||
this.redis = new Redis(redisUrl);
|
||||
this.startSync();
|
||||
}
|
||||
|
||||
private startSync() {
|
||||
setInterval(async () => {
|
||||
// Sync local counters to Redis
|
||||
const pipeline = this.redis.pipeline();
|
||||
|
||||
this.localCounter.forEach((count, key) => {
|
||||
if (count > 0) {
|
||||
pipeline.incrby(key, count);
|
||||
}
|
||||
});
|
||||
|
||||
await pipeline.exec();
|
||||
this.localCounter.clear();
|
||||
}, this.syncInterval);
|
||||
}
|
||||
|
||||
async checkLimit(
|
||||
key: string,
|
||||
limit: number,
|
||||
window: number
|
||||
): Promise<boolean> {
|
||||
// Quick local check (approximate)
|
||||
const localCount = this.localCounter.get(key) || 0;
|
||||
|
||||
// Get current count from Redis
|
||||
const redisCount = parseInt(await this.redis.get(key) || '0');
|
||||
const totalCount = redisCount + localCount;
|
||||
|
||||
if (totalCount >= limit) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Increment locally
|
||||
this.localCounter.set(key, localCount + 1);
|
||||
|
||||
// Set expiry if new key
|
||||
if (redisCount === 0) {
|
||||
await this.redis.setex(key, window, '0');
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rate Limit Dashboard
|
||||
|
||||
```tsx
|
||||
// components/RateLimitStatus.tsx
|
||||
'use client';
|
||||
|
||||
import { useState, useEffect } from 'react';
|
||||
|
||||
interface RateLimitStatus {
|
||||
endpoint: string;
|
||||
limit: number;
|
||||
used: number;
|
||||
remaining: number;
|
||||
resetsAt: Date;
|
||||
}
|
||||
|
||||
export function RateLimitStatus({ apiKey }: { apiKey: string }) {
|
||||
const [status, setStatus] = useState<RateLimitStatus[]>([]);
|
||||
|
||||
useEffect(() => {
|
||||
fetch(`/api/rate-limits/status?apiKey=${apiKey}`)
|
||||
.then(res => res.json())
|
||||
.then(setStatus);
|
||||
}, [apiKey]);
|
||||
|
||||
return (
|
||||
<div className="bg-white rounded-lg shadow p-6">
|
||||
<h2 className="text-lg font-semibold mb-4">Rate Limit Status</h2>
|
||||
|
||||
<div className="space-y-4">
|
||||
{status.map((item) => (
|
||||
<div key={item.endpoint} className="border-b pb-4">
|
||||
<div className="flex justify-between mb-2">
|
||||
<span className="font-medium">{item.endpoint}</span>
|
||||
<span className="text-gray-500">
|
||||
{item.remaining} / {item.limit} remaining
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="w-full bg-gray-200 rounded-full h-2">
|
||||
<div
|
||||
className={`h-2 rounded-full ${
|
||||
item.remaining < item.limit * 0.1
|
||||
? 'bg-red-500'
|
||||
: item.remaining < item.limit * 0.3
|
||||
? 'bg-yellow-500'
|
||||
: 'bg-green-500'
|
||||
}`}
|
||||
style={{ width: `${(item.remaining / item.limit) * 100}%` }}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<p className="text-sm text-gray-500 mt-1">
|
||||
Resets: {new Date(item.resetsAt).toLocaleTimeString()}
|
||||
</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
| Aspect | Recommendation |
|
||||
|--------|----------------|
|
||||
| **Algorithm** | Sliding window for accuracy |
|
||||
| **Storage** | Redis for distributed systems |
|
||||
| **Headers** | Always return limit headers |
|
||||
| **Granularity** | Per-user + per-IP |
|
||||
| **Grace Period** | Soft limits before hard block |
|
||||
| **Documentation** | Clear rate limit docs |
|
||||
|
||||
---
|
||||
|
||||
## Fazit
|
||||
|
||||
Rate Limiting erfordert:
|
||||
|
||||
1. **Algorithmus**: Sliding Window oder Token Bucket
|
||||
2. **Skalierung**: Redis für verteilte Systeme
|
||||
3. **Granularität**: User, IP, Endpoint-spezifisch
|
||||
4. **Transparenz**: Headers und Dokumentation
|
||||
|
||||
Rate Limiting schützt APIs und verbessert Fairness.
|
||||
|
||||
---
|
||||
|
||||
## Bildprompts
|
||||
|
||||
1. "Rate limiting algorithm comparison diagram, token bucket and sliding window"
|
||||
2. "API rate limit dashboard, usage gauges and quotas"
|
||||
3. "Distributed rate limiting architecture, Redis cluster"
|
||||
|
||||
---
|
||||
|
||||
## Quellen
|
||||
|
||||
- [Stripe Rate Limiting](https://stripe.com/blog/rate-limiters)
|
||||
- [Cloudflare Rate Limiting](https://blog.cloudflare.com/counting-things-a-lot-of-different-things/)
|
||||
- [Redis Rate Limiting Patterns](https://redis.io/commands/incr#pattern-rate-limiter)
|
||||
- [Kong Rate Limiting](https://docs.konghq.com/hub/kong-inc/rate-limiting/)
|
||||
Reference in New Issue
Block a user