Sync pi-tui Unicode input + tests, bump to 0.1.1

Author: steipete

CHANGELOG.md

@@ -0,0 +1,11 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [0.1.1] - 2025-11-17
+- Allow printable Unicode in editor paste path (drops only control characters) to match upstream pi-tui Unicode input behavior.
+- Add Unicode-focused editor tests (emoji, umlauts, cursor movement, control-char stripping).
+- Document printable Unicode handling and Alt+Enter newline guidance in spec/README.
+
+## [0.1.0] - 2025-11-15
+- Initial TauTUI sync with pi-tui core features (editor, autocomplete, markdown, loader, select list, renderer).

Package.swift

@@ -3,6 +3,9 @@
 
 import PackageDescription
 
+// Current package version
+private let packageVersion = "0.1.1"
+
 let package = Package(
     name: "TauTUI",
     platforms: [

README.md

@@ -145,7 +145,7 @@ editor.onSubmit = { print("Submitted", $0) }
 editor.setAutocompleteProvider(CombinedAutocompleteProvider(commands: [DemoCommand()]))
 ```
 
-Key bindings include Enter/Shift+Enter, Ctrl+K/U/W/A/E, Option word motion, Tab for autocomplete, Escape to cancel, etc.
+Key bindings include Enter for submit, Shift/Ctrl/Alt+Enter for newlines (Alt is the most reliable across terminals), Ctrl+K/U/W/A/E, Option word motion, Tab for autocomplete, Escape to cancel, etc.
 
 ### SelectList
 

Sources/TauTUI/Components/Editor.swift

@@ -329,8 +329,9 @@ public final class Editor: Component {
                 partial.append(char)
                 return
             }
-            guard let scalar = char.unicodeScalars.first else { return }
-            if scalar.value >= 32, scalar.value <= 126 {
+            // Keep any printable Unicode character; drop control chars (< 0x20).
+            let hasControl = char.unicodeScalars.contains { $0.value < 32 }
+            if !hasControl {
                 partial.append(char)
             }
         }
@@ -349,7 +350,7 @@ public final class Editor: Component {
             return
         }
         if lines.count == 1 {
-            for char in normalized {
+            for char in sanitized {
                 self.insertCharacter(String(char))
             }
             return

Tests/TauTUITests/EditorUnicodeTests.swift

@@ -0,0 +1,80 @@
+import Foundation
+import Testing
+@testable import TauTUI
+
+private func type(_ text: String, into editor: Editor) {
+    for char in text {
+        editor.handle(input: .key(.character(char)))
+    }
+}
+
+@Suite("Editor + Unicode")
+struct EditorUnicodeTests {
+    @Test
+    func insertsMixedUnicode() async throws {
+        let editor = Editor()
+        type("Hello äöü 😀", into: editor)
+        #expect(editor.getText() == "Hello äöü 😀")
+    }
+
+    @Test
+    func backspaceHandlesSingleAndMultiScalarCharacters() async throws {
+        let editor = Editor()
+        type("ä👍", into: editor)
+
+        editor.handle(input: .key(.backspace)) // remove 👍
+        #expect(editor.getText() == "ä")
+
+        editor.handle(input: .key(.backspace)) // remove ä
+        #expect(editor.getText().isEmpty)
+    }
+
+    @Test
+    func arrowNavigationAcrossEmoji() async throws {
+        let editor = Editor()
+        type("😀👍", into: editor)
+        editor.handle(input: .key(.arrowLeft))
+        editor.handle(input: .key(.character("x")))
+        #expect(editor.getText() == "😀x👍")
+    }
+
+    @Test
+    func insertAfterCursorMoveOverUmlauts() async throws {
+        let editor = Editor()
+        type("äöü", into: editor)
+        editor.handle(input: .key(.arrowLeft))
+        editor.handle(input: .key(.arrowLeft))
+        editor.handle(input: .key(.character("x")))
+        #expect(editor.getText() == "äxöü")
+    }
+
+    @Test
+    func pastePreservesUnicodeAndStripsControlChars() async throws {
+        let editor = Editor()
+        editor.handle(input: .paste("Hällö\u{0007} Wörld! 😀 äöüÄÖÜß"))
+        #expect(editor.getText() == "Hällö Wörld! 😀 äöüÄÖÜß")
+    }
+
+    @Test
+    func preservesUmlautsAcrossLineBreaks() async throws {
+        let editor = Editor()
+        type("äöü\nÄÖÜ", into: editor)
+        #expect(editor.getText() == "äöü\nÄÖÜ")
+    }
+
+    @Test
+    func setTextReplacesDocumentWithUnicode() async throws {
+        let editor = Editor()
+        editor.setText("Hällö Wörld! 😀 äöüÄÖÜß")
+        #expect(editor.getText() == "Hällö Wörld! 😀 äöüÄÖÜß")
+    }
+
+    @Test
+    func ctrlAMoveThenInsertWithUnicodePresent() async throws {
+        let editor = Editor()
+        type("äöü", into: editor)
+        editor.handle(input: .key(.character("a"), modifiers: [.control]))
+        editor.handle(input: .key(.character("X")))
+        #expect(editor.getText() == "Xäöü")
+    }
+}

docs/pitui-sync.md

@@ -0,0 +1,29 @@
+# pi-tui sync log (Nov 15–17, 2025)
+
+Context: track upstream changes in `packages/tui` from `pi-mono` since **2025-11-15** (last pre-window commit: `1afe40e` / v0.7.10). Current head inspected: `origin/main` as of **2025-11-17**.
+
+## Commits (newest → oldest)
+- **2025-11-16 23:08 CET — ed53fce — v0.7.13:** version bump after Unicode input fix (no code changes).
+- **2025-11-16 23:06 CET — a032d41 — merge fix/support-umlauts-and-unicode:** pulls in editor Unicode work + README tweak; keeps new tests.
+- **2025-11-16 23:05 CET — adc8b0e — refactor:** clarifies the Unicode filter by checking `charCodeAt(0) >= 32` for regular characters.
+- **2025-11-16 22:56 CET — b2491aa — v0.7.12:** version bump for models.json work (no TUI code touched).
+- **2025-11-16 21:05 CET — 7efcda6 — test(editor):** reorganizes Unicode tests for clarity.
+- **2025-11-16 21:01 CET — fd2b2ec — Filter model selector…:** only bumps `packages/tui/package.json`; logic elsewhere.
+- **2025-11-16 18:15 CET — 500e0f8 — test(editor):** adds coverage for umlauts, emojis, cursor movement over multi-code-unit chars, Backspace on surrogate pairs, setText with Unicode, and Ctrl+A insertion.
+- **2025-11-16 18:09 CET — efa6a00 — feat(tui):** functional change—editor now treats any `charCode >= 32` as insertable (was ASCII-only) and updates keybinding docs to note `Alt+Enter` as the most reliable “new line” chord.
+
+## Porting impact for TauTUI
+- **Input filtering:** `packages/tui/src/components/editor.ts` now allows all printable Unicode (`charCode >= 32`) and removes the upper ASCII bound in paste filtering. Our Swift `Editor` still limits pasted characters to scalars `<= 126` (see `Sources/TauTUI/Components/Editor.swift`, `handlePaste`), so Unicode input/paste parity is missing—needs to be relaxed to `>= 32` with no upper bound.
+- **Keybinding doc:** Upstream README calls out `Alt+Enter` as the most reliable way to insert a newline in terminals; mirror this note in our docs (`docs/spec.md` and/or README) when we next touch keybinding guidance.
+- **Tests to mirror:** Upstream added explicit Unicode scenarios. Add equivalent Swift tests covering:
+  - Mixed ASCII + umlaut + emoji insertion stays literal.
+  - Backspace over single-code-unit umlauts vs. multi-code-unit emojis (requiring two deletes).
+  - Cursor movement across multi-code-unit emojis before insertion.
+  - Unicode preserved across newlines and via `setText`/paste.
+  - Ctrl+A move-to-start followed by insertion with Unicode present.
+- **Version-only commits:** b2491aa/fd2b2ec/ed53fce are version bumps; no TauTUI action beyond tracking upstream tag alignment.
+
+## Next steps
+1) Update `Editor.handlePaste` (and any other printable-character guards) to accept all `>= 0x20` code points.  
+2) Add the Unicode-focused tests to `Tests/TauTUITests` to verify Backspace behavior on surrogate pairs.  
+3) Refresh newline keybinding wording to include `Alt+Enter` reliability note.

docs/spec.md

@@ -109,7 +109,7 @@ public protocol Terminal: AnyObject {
   - `PasteManager`: tracks bracketed paste state, large paste markers (`[paste #N +xx lines]`), Map from ID → content for substitution on submit.
   - `AutocompleteController`: wraps `AutocompleteProvider`, handles Tab triggers, slash command detection, forced file completion, `SelectList` overlay.
 - Rendering: draw horizontal lines (chalk gray equivalent) above/below; show fake cursor via reverse video; append autocomplete list when active (reusing `SelectList` rendering output).
-- Input pipeline replicates pi-tui order: bracketed paste start/end, autocomplete keys, Tab logic, control shortcuts (Ctrl+A/E/K/U/W, Option+Backspace, Shift+Enter combos), newline vs submit decision, backspace/delete/arrow keys, printable ASCII insertion.
+- Input pipeline replicates pi-tui order: bracketed paste start/end, autocomplete keys, Tab logic, control shortcuts (Ctrl+A/E/K/U/W, Option+Backspace, Shift+Enter combos), newline vs submit decision, backspace/delete/arrow keys, printable Unicode insertion (charCode ≥ 0x20).
 - Public API:
   ```swift
   final class Editor: Component {