Getting Started

1. Create new react app


npm create vite@latest my-app -- --template react-ts
cd ./my-app

2. Install dependencies


npm i three @react-three/fiber @react-three/drei
npm i face-tryon-core face-tryon-react

3. Place 3d model, occluder model, and task in public directory

Download Glasses Model. and place it in public directory of your code.
Download Ear Occluder. and place it in public directory of your code.
Download Face Landmarker Task and place it in public directory of your code.

Ear occluder model contains fake ears (two rectangles). They are invisible but occlude (block) parts of glasses behind them.

4. Add a few lines of code


import { Environment, Loader } from "@react-three/drei";
import { Canvas } from "@react-three/fiber";
import { CAMERA_POSITION, CAMERA_VERTICAL_FOV_DEGRESS } from "face-tryon-core";
import {
  useCameraFrameProvider,
  useFrameDimensions,
  useFaces,
  VideoBackground,
  CoverFit,
  FaceOccluder,
  GLTFFaceProp,
  GLTFOccluder,
  FaceMesh,
} from "face-tryon-react";
import { Suspense } from "react";
 
export function SimpleExample() {
  const { provider, video, setVideo } = useCameraFrameProvider();
  const { width, height } = useFrameDimensions(provider);
  const faces = useFaces(provider);
 
  return (
    <div style={{ position: "relative", width: "100%", padding: "1rem" }}>
      <video ref={setVideo} style={{ display: "none" }} autoPlay muted />
      <div style={{ width: "100%", height: 500 }}>
        <Canvas
          camera={{
            fov: CAMERA_VERTICAL_FOV_DEGRESS,
            position: CAMERA_POSITION,
            near: 0.01,
            far: 1000,
          }}
        >
          <Environment preset="studio" />
          <CoverFit width={width} height={height}>
            <VideoBackground
              frameDimensions={{ width, height }}
              image={video}
            />
            {faces.map((face, i) => (
              <Suspense key={i} fallback={null}>
                <FaceOccluder face={face} />
                <GLTFFaceProp face={face} modelPath="/glasses.glb" />
                <GLTFOccluder face={face} modelPath="/ear_occluder.glb" />
              </Suspense>
            ))}
          </CoverFit>
        </Canvas>
        <Loader containerStyles={{ backgroundColor: "rgba(0,0,0,0.2)" }} />
      </div>
    </div>
  );
}

Here is how the output should look like.

Since we only have front part of the facemesh with partial forehead, glasses would go through your head if you look at them from top.

Please allow camera access after clicking the button. Everything is processed on your device, no data leaves your device.