Initial commit: Portfolio Website

Vollständige Next.js 15 Portfolio-Website mit: - Blog-System mit 100+ Artikeln - Supabase-Integration - Responsive Design mit Tailwind CSS - TypeScript-Konfiguration - Testing-Setup mit Vitest und Playwright Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 15:07:20 +01:00
commit e1bbe5455d
315 changed files with 94124 additions and 0 deletions
@@ -0,0 +1,563 @@
+# Conversational AI 2.0: Natürliches Turn-Taking und Interruption Handling
+
+**Meta-Description:** Fortgeschrittene Techniken für natürliche Sprachinteraktion. Turn-Taking-Modelle, intelligente Unterbrechungserkennung und Full-Duplex-Kommunikation.
+
+**Keywords:** Conversational AI, Turn-Taking, Interruption Handling, Full-Duplex, Voice Agent, Natural Conversation, VAD, TRP
+
+---
+
+## Einführung
+
+Die größte Herausforderung für Voice AI ist nicht die Spracherkennung – es ist das **Timing**. Wann ist der User fertig? Wann darf der Agent sprechen? Wie reagiert man auf Unterbrechungen?
+
+> "OpenAIs neue Audio-Architektur 2026 zielt auf niedrigere Latenz und natürlicheres Back-and-Forth ab – weg vom Walkie-Talkie-Modell."
+
+---
+
+## Das Problem mit Silence-Based Detection
+
+### Warum einfache Stille nicht funktioniert
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│              SILENCE-BASED VS. INTELLIGENT DETECTION        │
+├─────────────────────────────────────────────────────────────┤
+│                                                             │
+│  Silence-Based:                                             │
+│  User: "Ich möchte..." [500ms Pause] "...einen Kaffee"     │
+│  Agent: [Unterbricht bei Pause] "Wie kann ich helfen?"     │
+│  ❌ Frustrierend für User                                   │
+│                                                             │
+│  Intelligent Turn-Taking:                                   │
+│  User: "Ich möchte..." [500ms Pause] "...einen Kaffee"     │
+│  Agent: [Wartet auf semantisches Ende]                      │
+│  User: "...mit Milch."                                      │
+│  Agent: "Einen Kaffee mit Milch, kommt sofort!"            │
+│  ✅ Natürliche Konversation                                 │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Transition-Relevance Places (TRPs)
+
+Natürliche Gespräche haben **Übergangspunkte**, die nicht nur durch Stille definiert sind:
+
+| Signal | Beispiel | Zuverlässigkeit |
+|--------|----------|-----------------|
+| **Syntaktisch** | Vollständiger Satz | Hoch |
+| **Prosodisch** | Fallende Intonation | Mittel |
+| **Pragmatisch** | Frage gestellt | Hoch |
+| **Semantisch** | Gedanke abgeschlossen | Mittel |
+| **Stille** | > 700ms Pause | Niedrig |
+
+---
+
+## Turn-Taking Architektur
+
+```typescript
+// src/turn-taking/detector.ts
+interface TurnTakingSignals {
+  // Audio-basiert
+  silenceDurationMs: number;
+  pitchContour: 'rising' | 'falling' | 'flat';
+  speechRate: number;
+
+  // Text-basiert (von ASR)
+  lastUtterance: string;
+  isQuestion: boolean;
+  isComplete: boolean;
+
+  // Kontext
+  conversationHistory: Message[];
+  expectedResponseType: 'answer' | 'continuation' | 'acknowledgment';
+}
+
+interface TurnDecision {
+  action: 'wait' | 'respond' | 'backchannel';
+  confidence: number;
+  reasoning: string;
+}
+
+class IntelligentTurnDetector {
+  private model: TurnPredictionModel;
+
+  constructor() {
+    this.model = new TurnPredictionModel();
+  }
+
+  async predict(signals: TurnTakingSignals): Promise<TurnDecision> {
+    // Multi-Signal-Analyse
+    const features = this.extractFeatures(signals);
+
+    // Heuristiken für schnelle Entscheidungen
+    if (signals.silenceDurationMs > 2000) {
+      return { action: 'respond', confidence: 0.95, reasoning: 'Long silence' };
+    }
+
+    if (signals.isQuestion && signals.silenceDurationMs > 300) {
+      return { action: 'respond', confidence: 0.9, reasoning: 'Question asked' };
+    }
+
+    // ML-basierte Prediction für komplexe Fälle
+    const prediction = await this.model.predict(features);
+
+    return prediction;
+  }
+
+  private extractFeatures(signals: TurnTakingSignals) {
+    return {
+      // Normalisierte Features für ML
+      silenceNormalized: Math.min(signals.silenceDurationMs / 2000, 1),
+      pitchFalling: signals.pitchContour === 'falling' ? 1 : 0,
+      syntacticCompleteness: this.analyzeSyntax(signals.lastUtterance),
+      semanticCompleteness: this.analyzeSemantics(signals.lastUtterance),
+      questionProbability: signals.isQuestion ? 1 : 0,
+      // ...weitere Features
+    };
+  }
+
+  private analyzeSyntax(utterance: string): number {
+    // Einfache Heuristik: Endet mit Satzzeichen?
+    if (/[.!?]$/.test(utterance.trim())) return 1;
+
+    // Unvollständiger Satz
+    if (/\b(und|aber|oder|weil|dass)\s*$/.test(utterance)) return 0;
+
+    return 0.5;
+  }
+
+  private analyzeSemantics(utterance: string): number {
+    // Kann mit LLM verbessert werden
+    const incompletePatterns = [
+      /ich möchte\s*$/i,
+      /ich brauche\s*$/i,
+      /können Sie\s*$/i,
+      /wie wäre es\s*$/i
+    ];
+
+    for (const pattern of incompletePatterns) {
+      if (pattern.test(utterance)) return 0;
+    }
+
+    return 0.7;
+  }
+}
+```
+
+---
+
+## Interruption Handling
+
+### Typen von Unterbrechungen
+
+```typescript
+enum InterruptionType {
+  // Kooperativ
+  AGREEMENT = 'agreement',        // "Ja, genau!"
+  ASSISTANCE = 'assistance',      // User hilft Agent beim Formulieren
+  CLARIFICATION = 'clarification', // "Moment, was meinen Sie?"
+
+  // Disruptiv
+  DISAGREEMENT = 'disagreement',  // "Nein, das stimmt nicht"
+  TOPIC_CHANGE = 'topic_change',  // "Egal, andere Frage..."
+  CORRECTION = 'correction',      // "Nicht 3, ich sagte 5"
+
+  // Technisch
+  BARGE_IN = 'barge_in'          // User will Agent stoppen
+}
+
+interface InterruptionEvent {
+  type: InterruptionType;
+  timestamp: number;
+  userUtterance: string;
+  agentWasSpeaking: boolean;
+  agentUtteranceProgress: number; // 0-1
+}
+```
+
+### Intelligente Reaktionen
+
+```typescript
+// src/turn-taking/interruption-handler.ts
+class InterruptionHandler {
+  async handleInterruption(
+    event: InterruptionEvent,
+    agent: VoiceAgent
+  ): Promise<void> {
+    // 1. Agent sofort stoppen
+    await agent.stopSpeaking();
+
+    // 2. Typ klassifizieren
+    const type = await this.classifyInterruption(event);
+
+    // 3. Entsprechend reagieren
+    switch (type) {
+      case InterruptionType.AGREEMENT:
+        // Kurz bestätigen, dann weitermachen
+        await agent.say("Genau.");
+        await agent.continueFromLastPoint();
+        break;
+
+      case InterruptionType.CLARIFICATION:
+        // Erklärung geben
+        await agent.say("Lass mich das erklären...");
+        await agent.clarifyLastStatement();
+        break;
+
+      case InterruptionType.CORRECTION:
+        // Korrektur akzeptieren
+        await agent.say("Entschuldigung, ich korrigiere...");
+        await agent.processUserInput(event.userUtterance);
+        break;
+
+      case InterruptionType.BARGE_IN:
+        // Komplett neuen Input verarbeiten
+        await agent.processUserInput(event.userUtterance);
+        break;
+
+      case InterruptionType.TOPIC_CHANGE:
+        // Context wechseln
+        await agent.say("Okay, zum neuen Thema...");
+        await agent.processUserInput(event.userUtterance);
+        break;
+    }
+  }
+
+  private async classifyInterruption(
+    event: InterruptionEvent
+  ): Promise<InterruptionType> {
+    // Schnelle Heuristiken
+    const text = event.userUtterance.toLowerCase();
+
+    if (/^(ja|genau|richtig|stimmt)/.test(text)) {
+      return InterruptionType.AGREEMENT;
+    }
+
+    if (/^(nein|falsch|nicht|stop)/.test(text)) {
+      return InterruptionType.DISAGREEMENT;
+    }
+
+    if (/^(was|wie|warum|moment)/.test(text)) {
+      return InterruptionType.CLARIFICATION;
+    }
+
+    // LLM für komplexere Fälle
+    return await this.classifyWithLLM(event);
+  }
+}
+```
+
+---
+
+## Full-Duplex Communication
+
+### Das NVIDIA PersonaPlex Konzept
+
+```typescript
+// Full-Duplex: Agent kann gleichzeitig hören und sprechen
+class FullDuplexAgent {
+  private isSpeaking = false;
+  private isListening = true; // Immer an
+  private audioBuffer: Buffer[] = [];
+
+  async processAudioStream(
+    input: AsyncIterable<Buffer>,
+    output: (chunk: Buffer) => void
+  ) {
+    // Parallele Verarbeitung
+    const [transcription, speechOutput] = await Promise.all([
+      this.transcribeStream(input),
+      this.generateSpeech()
+    ]);
+
+    // Während Agent spricht, weiter zuhören
+    for await (const audioChunk of input) {
+      // VAD prüfen
+      if (this.detectVoiceActivity(audioChunk)) {
+        // User spricht während Agent spricht
+        if (this.isSpeaking) {
+          await this.handleOverlap(audioChunk);
+        }
+      }
+    }
+  }
+
+  private async handleOverlap(userAudio: Buffer) {
+    // Analyse: Ist es eine Unterbrechung?
+    const energy = this.calculateEnergy(userAudio);
+
+    if (energy > this.thresholdForInterruption) {
+      // Lautstärke des Agents reduzieren
+      this.reduceAgentVolume(0.3);
+
+      // Warten ob User weiterspricht
+      await this.waitForUserIntent(500);
+
+      if (this.userContinuesSpeaking) {
+        // Komplett stoppen
+        await this.stopSpeaking();
+      } else {
+        // War nur Backchannel, weitermachen
+        this.restoreAgentVolume();
+      }
+    }
+  }
+}
+```
+
+---
+
+## Backchannel Responses
+
+### Natürliche Bestätigungen während User spricht
+
+```typescript
+interface BackchannelConfig {
+  enabled: boolean;
+  responses: string[];
+  triggerInterval: number; // ms
+  maxPerTurn: number;
+}
+
+class BackchannelGenerator {
+  private config: BackchannelConfig = {
+    enabled: true,
+    responses: ['Mhm', 'Ja', 'Verstehe', 'Okay', 'Aha'],
+    triggerInterval: 3000,
+    maxPerTurn: 3
+  };
+
+  private backchannelCount = 0;
+  private lastBackchannel = 0;
+
+  shouldGenerateBackchannel(
+    signals: TurnTakingSignals
+  ): { should: boolean; response: string } {
+    const now = Date.now();
+
+    // Limits prüfen
+    if (this.backchannelCount >= this.config.maxPerTurn) {
+      return { should: false, response: '' };
+    }
+
+    if (now - this.lastBackchannel < this.config.triggerInterval) {
+      return { should: false, response: '' };
+    }
+
+    // Trigger-Bedingungen
+    const shouldTrigger =
+      signals.silenceDurationMs > 200 &&
+      signals.silenceDurationMs < 500 &&
+      !signals.isComplete &&
+      signals.lastUtterance.length > 20;
+
+    if (shouldTrigger) {
+      this.backchannelCount++;
+      this.lastBackchannel = now;
+
+      const response = this.selectResponse(signals);
+      return { should: true, response };
+    }
+
+    return { should: false, response: '' };
+  }
+
+  private selectResponse(signals: TurnTakingSignals): string {
+    // Kontext-abhängige Auswahl
+    if (signals.lastUtterance.includes('Problem')) {
+      return 'Oh je';
+    }
+
+    if (signals.lastUtterance.includes('?')) {
+      return 'Mhm';
+    }
+
+    // Random für Variation
+    const idx = Math.floor(Math.random() * this.config.responses.length);
+    return this.config.responses[idx];
+  }
+
+  resetForNewTurn() {
+    this.backchannelCount = 0;
+  }
+}
+```
+
+---
+
+## Production-Ready Implementation
+
+```typescript
+// src/voice-agent-v2.ts
+import { DeepgramStreamer } from './services/deepgram';
+import { ElevenLabsStreamer } from './services/elevenlabs';
+import { IntelligentTurnDetector } from './turn-taking/detector';
+import { InterruptionHandler } from './turn-taking/interruption-handler';
+import { BackchannelGenerator } from './turn-taking/backchannel';
+
+export class ConversationalVoiceAgent {
+  private deepgram = new DeepgramStreamer();
+  private elevenlabs = new ElevenLabsStreamer();
+  private turnDetector = new IntelligentTurnDetector();
+  private interruptionHandler = new InterruptionHandler();
+  private backchannel = new BackchannelGenerator();
+
+  private state: 'listening' | 'processing' | 'speaking' = 'listening';
+  private currentUtterance = '';
+
+  async start(
+    audioInput: AsyncIterable<Buffer>,
+    onAudioOutput: (chunk: Buffer) => void
+  ) {
+    await this.deepgram.startStream(
+      {
+        model: 'nova-2',
+        language: 'de',
+        smart_format: true,
+        interim_results: true,
+        endpointing: 300 // Schnelles Feedback
+      },
+      async (text, isFinal) => {
+        this.currentUtterance = text;
+
+        // Interim: Prüfe auf Unterbrechung
+        if (!isFinal && this.state === 'speaking') {
+          await this.interruptionHandler.handleInterruption(
+            {
+              type: InterruptionType.BARGE_IN,
+              timestamp: Date.now(),
+              userUtterance: text,
+              agentWasSpeaking: true,
+              agentUtteranceProgress: 0.5
+            },
+            this
+          );
+          return;
+        }
+
+        // Final: Turn-Taking-Entscheidung
+        if (isFinal) {
+          const decision = await this.turnDetector.predict({
+            silenceDurationMs: 0,
+            pitchContour: 'falling',
+            speechRate: 1,
+            lastUtterance: text,
+            isQuestion: text.includes('?'),
+            isComplete: true,
+            conversationHistory: [],
+            expectedResponseType: 'answer'
+          });
+
+          if (decision.action === 'respond') {
+            this.state = 'processing';
+            await this.generateResponse(text, onAudioOutput);
+          }
+        } else {
+          // Backchannel prüfen
+          const bc = this.backchannel.shouldGenerateBackchannel({
+            silenceDurationMs: 300,
+            pitchContour: 'flat',
+            speechRate: 1,
+            lastUtterance: text,
+            isQuestion: false,
+            isComplete: false,
+            conversationHistory: [],
+            expectedResponseType: 'continuation'
+          });
+
+          if (bc.should) {
+            // Leises Backchannel ohne State-Wechsel
+            await this.speakQuietly(bc.response, onAudioOutput);
+          }
+        }
+      }
+    );
+
+    for await (const chunk of audioInput) {
+      this.deepgram.sendAudio(chunk);
+    }
+  }
+
+  private async speakQuietly(
+    text: string,
+    onOutput: (chunk: Buffer) => void
+  ) {
+    // Niedrige Lautstärke für Backchannel
+    for await (const chunk of this.elevenlabs.streamSpeech(text, {
+      voiceId: 'default',
+      modelId: 'eleven_flash_v2_5',
+      stability: 0.3,
+      similarityBoost: 0.5,
+      latencyOptimization: 4
+    })) {
+      onOutput(this.reduceVolume(chunk, 0.4));
+    }
+  }
+
+  private reduceVolume(audio: Buffer, factor: number): Buffer {
+    // PCM volume reduction
+    const samples = new Int16Array(audio.buffer);
+    for (let i = 0; i < samples.length; i++) {
+      samples[i] = Math.round(samples[i] * factor);
+    }
+    return Buffer.from(samples.buffer);
+  }
+
+  async stopSpeaking() {
+    // Implementation für sofortigen Stop
+    this.state = 'listening';
+  }
+
+  private async generateResponse(
+    input: string,
+    onOutput: (chunk: Buffer) => void
+  ) {
+    this.state = 'speaking';
+    // LLM + TTS Pipeline...
+    this.state = 'listening';
+    this.backchannel.resetForNewTurn();
+  }
+}
+```
+
+---
+
+## Metriken für Turn-Taking Qualität
+
+| Metrik | Zielwert | Beschreibung |
+|--------|----------|--------------|
+| **Turn-Taking Latenz** | < 500ms | Zeit von User-Ende bis Agent-Start |
+| **False Interruptions** | < 5% | Agent unterbricht User fälschlich |
+| **Missed TRPs** | < 10% | Agent verpasst Übergangspunkte |
+| **Interruption Recovery** | < 1s | Zeit bis normale Konversation |
+
+---
+
+## Fazit
+
+Natürliches Turn-Taking erfordert:
+
+1. **Multi-Signal-Analyse**: Nicht nur Stille, sondern Syntax + Semantik + Prosodie
+2. **Intelligente Unterbrechungserkennung**: Kooperativ vs. Disruptiv unterscheiden
+3. **Backchannel-Responses**: Aktives Zuhören signalisieren
+4. **Full-Duplex**: Gleichzeitig hören und sprechen können
+
+Das Walkie-Talkie-Modell ist Geschichte.
+
+---
+
+## Bildprompts
+
+1. "Two people in natural conversation with speech bubbles overlapping, timing visualization, warm illustration style"
+2. "AI voice assistant with sound waves showing bidirectional flow, full-duplex concept, modern tech art"
+3. "Conversation flow diagram with turn-taking points marked, linguistic analysis visualization"
+
+---
+
+## Quellen
+
+- [NVIDIA PersonaPlex Research](https://research.nvidia.com/labs/adlr/personaplex/)
+- [Amazon Nova 2 Sonic Announcement](https://aws.amazon.com/blogs/aws/introducing-amazon-nova-2-sonic-next-generation-speech-to-speech-model-for-conversational-ai/)
+- [Retell AI Turn-Taking Model](https://www.retellai.com/blog/how-retell-ais-turn-taking-model-ensures-seamless-calls)
+- [Interruption Handling Research (arXiv)](https://arxiv.org/html/2501.01568v1)
+- [Turn-Taking in Conversational Systems (MDPI)](https://www.mdpi.com/2227-7080/13/12/591)