Sketch mode ideas

[andrewvarga]

I wanted to have a document where I'm putting my ideas about the upcoming implementation of the new sketch mode, focusing on sceneEntities / three.js / general architecture not so much on the constraints side.
These are things I saw in the existing implementation and I thought it would be worth to discuss.

• More modular setup: the existing entity types, tools are usually put into the same file and their unique implementations are often selected using conditional branches. This results in some very large files, like SceneEntities being 4K+ lines, ModelingMachine 5K.
  I think it would be nice to have each entity / tool in its own file. Now removing or adding an entity / tool includes touching multiple files and it's easy to forget a few locations, I think the new sketch mode is already going into this direction though?
• It feels three.js objects are a bit too tightly coupled to our data. There is a lot of info in userData and we don't really have a chance to swap three.js to another engine. And while we may never have to do that in practice, I think there is some value in making it more encapsulated.
• Reacting to changes often happens locally. Eg. updateSceneObjectColors which sets the selected segments to blue when the selection happens. I think it would simplify a lot if we instead had a minimal state being the source of truths and rendering would reflect that state by design without needing to manually update objects. I think each frame we should update the three.js render objects according to their state in a more "immediate mode rendering style" instead of updating them on the fly when they change.
  Eg. there is a updateSceneObjectColors function which updates three.js material properties based on the current selection, and it is invoked in ModelingMachine when selection changes. If instead we set these properties before each render from the current state it would be impossible to miss and rendering would always be consistent.
  In a more immediate more paradigm things could look like (simplified code which misses some details):

 interface State {
  entities: IEntity[]
  selected: IEntity[]
  hovering: IEntity | undefined
}

function drawStage(state: State) {
  for (const entity of entities) {
    if (entity.visible) {
      drawEntity(entity)
    }
  }
}
function drawEntity(entity: IEntity) {
  // ...
}

• Intersection calculation: now we use a raycastRing method which runs the raycaster multiple times to accomodate for the objects being too small at times. It would be faster and more deterministic to use larger bounding boxes for objects hit area or potentially switch to a pixel based picking (although that's not really needed until we have transparent objects).
• Use of Singletons: these global singleton managers like kclManager, codeManager, sceneInfra, etc could be turned into simple instances that are created ones and passed to where they are needed through dependency injection. [This may already be in progress]
• XState: I'm just mentioning this for completeness, since this is a long shot and probably too late. I think xstate brings a lot of good things, a visual state machine with a GUI, solid conventions to stick to, but it also brings its own gotchas and is a bit of a black box. To me it seems dealing with async actions that can throw errors potentially is often gnarly and can have surprising behaviours stemming from incorrect assumptions from our end. I've seen errors being suppressed by xstate and it's not always clear (to me) what happens in case there is an error in an async action / actor's execution, etc.
  To me there is a lot of value in owning and keeping our code as simple as possible which is easiest if we own it as much as possible. Also, without xstate we'd need something like redux or our own state manager which bring their own problems..maybe the solution is to use xstate better instead of not using it.

Example of how this would look like with a custom useModel implementation.

// useModel.ts
import { useState, useCallback, useMemo } from "react"

export interface Tool {
	name: string
	activate: () => void
	deactivate: () => void
}

export function useModel(initialTools: Tool[] = []) {
	const [tools] = useState<Tool[]>(initialTools)
	const [activeTool, setActiveTool] = useState<Tool | undefined>(undefined)

	const changeTool = useCallback((tool: Tool) =>
	{
		setActiveTool(prev =>
		{
			if (prev !== tool)
			{
				prev?.deactivate()
				tool.activate()
				return tool
			}
			return prev
		})
	}, [])

	return useMemo(() => ({
		tools,
		activeTool,
		changeTool,
	}), [tools, activeTool, changeTool])
}

// Toolbar.tsx
import React from "react"
import { useModel } from "./useModel"

interface ToolButtonProps {
	tool: any
	active: boolean
	onClick: () => void
}

function ToolButton({ tool, active, onClick }: ToolButtonProps) {
	return (
		<button
			className={`ToolButton ${active ? "active" : ""}`}
			onClick={onClick}
		>
			{tool.name}
		</button>
	)
}

export function ToolBar() {
	const model = useModel([
		{ name: "Move", activate: () => console.log("Move on"), deactivate: () => console.log("Move off") },
		{ name: "Rotate", activate: () => console.log("Rotate on"), deactivate: () => console.log("Rotate off") },
		{ name: "Scale", activate: () => console.log("Scale on"), deactivate: () => console.log("Scale off") },
	])

	return (
		<div className="Toolbar">
			{model.tools.map(tool => (
				<ToolButton
					key={tool.name}
					tool={tool}
					active={model.activeTool === tool}
					onClick={() => model.changeTool(tool)}
				/>
			))}
		</div>
	)
}

Obviously I'm missing some details but if it's our own code things can be significantly simpler compared to using a general state framework which has to accommodate for many different types of applications, not just one.