Fabric.js mobile text input

Fabric.js types through a hidden textarea. On a phone that works poorly (fabric.js #6588), so Fabric.js mobile support for text needs extra work:

An Android keyboard sends key code 229 for every key, so Fabric cannot see which key was pressed.
The keyboard changes text away from the cursor: autocorrect replaces the word before it, and a suggestion replaces the word being typed.
A swipe on the space bar moves the cursor without any key event.
iOS zooms the page when the textarea, with its 1px font, gets focus.

Fabric works out each edit from its own idea of the cursor, so after an autocorrect the letters, or their styles, land in the wrong place in an IText or Textbox.

Try it

Simulate an Android keyboard fixing “im” to “I'm” before the cursor.

Loading the live demo. Its code is below.

mobile-input-demo.tsx

"use client";
 
import { type Canvas, IText } from "fabric";
import {
  attachMobileTextInput,
  type MobileTextInput,
} from "fabricjs-document-engine/text";
import { RotateCcwIcon, SmartphoneIcon, SpellCheckIcon } from "lucide-react";
import { useTranslations } from "next-intl";
import { useEffect, useRef, useState } from "react";
 
import { DemoCanvas, DemoFrame, Status, Toolbar, ToolButton } from "./demo-ui";
import { useFabricCanvas } from "./use-fabric-canvas";
 
const BOLD = { fontWeight: "bold", fill: "#6f00ff" } as const;
 
function makeText() {
  return new IText("im here", {
    left: 40,
    top: 110,
    originX: "left",
    originY: "top",
    fontSize: 44,
    fill: "#0d0d0d",
    styles: { 0: { 3: BOLD, 4: BOLD, 5: BOLD, 6: BOLD } },
  });
}
 
function boldLetters(text: IText) {
  return [...text.text]
    .filter((_, index) => text.styles[0]?.[index]?.fontWeight === "bold")
    .join("");
}
 
export default function MobileInputDemo() {
  const t = useTranslations("demos");
  const { elementRef, canvas } = useFabricCanvas();
  const adapterRef = useRef<MobileTextInput | null>(null);
  const textRef = useRef<IText | null>(null);
  const [adapter, setAdapter] = useState(true);
  const [result, setResult] = useState<{ text: string; bold: string } | null>(
    null
  );
 
  function reset(target: Canvas) {
    if (textRef.current) target.remove(textRef.current);
    const text = makeText();
    textRef.current = text;
    target.add(text);
    target.requestRenderAll();
    setResult(null);
  }
 
  useEffect(() => {
    if (!canvas) return;
    if (adapter) adapterRef.current = attachMobileTextInput(canvas);
    reset(canvas);
    return () => {
      adapterRef.current?.detach();
      adapterRef.current = null;
    };
  }, [canvas, adapter]);
 
  function autocorrect() {
    const text = textRef.current;
    if (!canvas || !text) return;
    canvas.setActiveObject(text);
    text.enterEditing();
    text.setSelectionStart(text.text.length);
    text.setSelectionEnd(text.text.length);
    const textarea = text.hiddenTextarea;
    if (!textarea) return;
    textarea.dispatchEvent(
      new KeyboardEvent("keydown", {
        keyCode: 229,
        key: "Unidentified",
      } as KeyboardEventInit)
    );
    textarea.value = "I'm here";
    textarea.setSelectionRange(8, 8);
    textarea.dispatchEvent(
      new InputEvent("input", {
        inputType: "insertReplacementText",
        bubbles: true,
      })
    );
    text.exitEditing();
    canvas.requestRenderAll();
    setResult({ text: text.text, bold: boldLetters(text) });
  }
 
  return (
    <DemoFrame>
      <DemoCanvas elementRef={elementRef} label={t("canvas")} />
 
      <Toolbar>
        <ToolButton
          icon={SmartphoneIcon}
          label={adapter ? t("mobile.on") : t("mobile.off")}
          primary={adapter}
          onClick={() => setAdapter((value) => !value)}
        />
        <ToolButton
          icon={SpellCheckIcon}
          label={t("mobile.autocorrect")}
          disabled={!canvas}
          onClick={autocorrect}
        />
        <ToolButton
          icon={RotateCcwIcon}
          label={t("mobile.reset")}
          disabled={!canvas}
          onClick={() => canvas && reset(canvas)}
        />
      </Toolbar>
 
      <Status tone={!result ? "idle" : result.bold === "here" ? "ok" : "error"}>
        {result
          ? t("mobile.result", { text: result.text, bold: result.bold || "—" })
          : t("mobile.hint")}
      </Status>
    </DemoFrame>
  );
}

The button does what an Android keyboard does when it fixes "im" to "I'm": it changes the text before the cursor. With the adapter off, the bold style moves onto the wrong letters.

Turn it on

import { attachMobileTextInput } from "fabricjs-document-engine/text";
 
const mobileInput = attachMobileTextInput(canvas);
 
mobileInput.detach();

Attach it once, before the user starts editing. It works for every IText, Textbox and their subclasses on the canvas, including ones added later.

Each edit is read from the text itself: the shared start and end of the old and new text leave exactly what changed. Styles move with their letters.
The cursor follows the textarea, including moves the Android keyboard makes on its own.
The textarea gets focus inside the tap, so the keyboard opens on iOS and Android.
The textarea uses a 16px font, so iOS does not zoom.
When the keyboard opens and the page scrolls, the textarea stays at the cursor, so the input method shows its suggestions in the right place.

detach() puts Fabric's own handling back.

Keyboard options

attachMobileTextInput(canvas, {
  inputMode: "text",
  enterKeyHint: "done",
  autocapitalize: "sentences",
  autocorrect: true,
});

Option	Default
`inputMode`	`"text"`
`enterKeyHint`	`"enter"`
`autocapitalize`	`"sentences"`
`autocorrect`	`true`
`spellcheck`	`false`

Fabric turns autocorrect off because its own handling cannot apply corrections. With the adapter, autocorrect is safe to turn back on. For edits made from code, see Text styles from code. Every type is in the Text API, and this completes Fabric.js mobile support for text.

Typing on phones

Try it

Turn it on

Keyboard options