README.md

metadata

title: RMScript Web Demo
emoji: 🤖
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

RMScript Web Demo

Web-based IDE for writing, verifying, and executing rmscript code on Reachy Mini.

Architecture

Components

• Frontend (frontend/): Pure HTML/CSS/JavaScript web IDE

  • index.html: UI layout with embedded CSS styling
  • app.js: Application logic, compilation API calls, robot communication
  • Auto-detects environment (file:// for local, web server for production)

• Backend (backend/): FastAPI server (Python 3.11)

  • app.py: REST API for rmscript compilation and validation
  • Serves frontend files via static file mounting
  • Runs on port 8001 (local) or port 7860 (HuggingFace Spaces)

• Robot Daemon: Reachy Mini daemon (expected on localhost:8000)

  • REST API: /api/move/goto (with duration support)
  • WebSocket API: /api/move/ws/set_target (streaming, no duration)
  • State API: /api/state/* (robot current state)

Communication Flow

1. Script Verification/Compilation:

   • Frontend → Backend (/api/verify or /api/compile)
   • Backend compiles rmscript using the rmscript package
   • Returns IR (Intermediate Representation) + errors/warnings

2. Script Execution (Local):

   • Frontend → Robot Daemon (POST /api/move/goto)
   • Each IR action sent with duration and interpolation mode
   • Proper timing with minimum-jerk interpolation

3. Script Execution (HuggingFace Spaces):

   • Frontend → Robot Daemon (WebSocket /api/move/ws/set_target)
   • Fallback due to HTTPS → HTTP localhost blocking
   • No duration control - moves are instant

Quick Start

On HuggingFace Spaces

Just open the Space - the web IDE will load automatically.

Requirements:

• Reachy Mini daemon running on localhost:8000 on your local machine
• Modern web browser with WebSocket support

Local Development

1. Install rmscript package:

cd rmscript
uv sync

2. Start backend server:

cd backend
uv sync
uv run python app.py

3. Make sure your Reachy Mini daemon is running on localhost:8000

4. Open frontend/index.html in your web browser

Features

• Code Editor: Write rmscript with syntax examples
• Verify Button: Check script validity without executing
• Execute Button: Compile and run script on connected robot
• Live Compilation: See IR (Intermediate Representation) of your script
• Example Scripts: Load pre-written examples (basic, complex, repeat)
• Real-time Logging: Console shows compilation errors, warnings, and execution progress

Usage

1. Write rmscript in the editor (or load an example)
2. Click "Verify" to check syntax and semantics
3. Review any errors/warnings in the console
4. Click "Execute" to run on the robot (robot must be connected)
5. Watch execution progress in the Execution Info panel

Current Limitations

• Sound and picture actions are not yet implemented in execution
• Movement timing on HuggingFace Spaces: Due to browser security (HTTPS cannot fetch http://localhost), the Space version cannot use the robot's /goto endpoint which supports duration control.
  • On Spaces: Robot moves to each target as fast as possible (instant), but script execution pauses for the specified duration before the next move
  • Locally: Robot moves smoothly with proper duration and minimum-jerk interpolation
  • Recommendation: For demos requiring accurate movement timing, run locally

Deployment to HuggingFace Spaces

To deploy this demo to HuggingFace Spaces:

1. Prepare the repository structure:

   • Copy the rmscript and reachy_mini directories (without .git and binary files):

     cd rmscript_space_demo
     
     # Option 1: Using rsync (recommended - cleaner)
     rsync -av --exclude='.git' --exclude='.venv' --exclude='__pycache__' \
           --exclude='*.pyc' ../rmscript/ ./rmscript/
     rsync -av --exclude='.git' --exclude='.venv' --exclude='__pycache__' \
           --exclude='*.pyc' ../reachy_mini/ ./reachy_mini/
     
     # Option 2: Using cp then cleanup
     cp -r ../rmscript ./rmscript
     cp -r ../reachy_mini ./reachy_mini
     rm -rf ./rmscript/.git ./reachy_mini/.git
     rm -rf ./rmscript/.venv ./reachy_mini/.venv
     
     # Remove binary files that HuggingFace Spaces rejects
     rm -rf ./reachy_mini/src/reachy_mini/assets/*.wav
     rm -rf ./reachy_mini/src/reachy_mini/daemon/app/dashboard/static/fonts/*.ttf
     rm -rf ./reachy_mini/docs ./reachy_mini/examples
     rm -rf ./reachy_mini/src/reachy_mini/descriptions/reachy_mini/mjcf
     rm -rf ./rmscript/examples ./rmscript/tests
     

   • Your structure should be:

     rmscript_space_demo/
       ├── Dockerfile
       ├── README.md
       ├── .dockerignore
       ├── backend/
       ├── frontend/
       ├── rmscript/     ← copied source (no .git, no binaries)
       └── reachy_mini/  ← copied source (no .git, no binaries)
     

2. Create a new Space:

   • Go to https://huggingface.co/new-space
   • Select "Docker" as the SDK
   • Push this directory to your Space repository

3. The Space will:

   • Build the Docker container
   • Install rmscript and dependencies
   • Serve the web IDE on port 7860

Note: The Space connects to your local robot daemon on localhost:8000. Make sure it's running when using the demo.

Development

File Structure

rmscript_space_demo/
├── Dockerfile              # Docker build config for HuggingFace Spaces
├── .dockerignore          # Excludes .git, .venv, __pycache__, etc.
├── README.md              # This file (includes HF Space metadata in YAML)
├── backend/
│   ├── app.py            # FastAPI server (compilation + static file serving)
│   └── pyproject.toml    # Python dependencies
└── frontend/
    ├── index.html        # Web IDE UI (with embedded CSS)
    └── app.js            # Frontend logic (compilation + execution)

Key Implementation Details

Matrix to Euler Conversion (frontend/app.js:328-355):

• Converts 4x4 pose matrices (from rmscript IR) to euler angles (roll, pitch, yaw)
• Uses XYZ euler convention to match scipy.spatial.transform.Rotation.as_euler("xyz")
• Required because WebSocket API expects {x, y, z, roll, pitch, yaw} format

Environment Detection (frontend/app.js:9-11):

• Detects file:// protocol → uses http://localhost:8001 for backend
• Detects web server → uses relative URLs (same origin)
• Script loading path adapts similarly for app.js imports

Execution Strategy (frontend/app.js:357-427):

• Tries POST /api/move/goto with duration (works locally)
• Catches fetch failures → falls back to WebSocket (for Spaces)
• Logs warnings when fallback is used

Modifying the Demo

Frontend Changes:

• Edit frontend/app.js for logic changes
• Edit frontend/index.html for UI/styling changes
• Test by opening index.html directly in browser (local mode)

Backend Changes:

• Edit backend/app.py for API changes
• Run locally: cd backend && uv run python app.py
• Test API at http://localhost:8001/docs (FastAPI Swagger UI)

Deployment:

• Changes auto-deploy when pushed to HuggingFace Space repo
• Build logs visible in Space's "Build" tab
• Runtime logs visible in Space's "Logs" tab

Technical Notes

Browser Security & Mixed Content

The Problem:

• HuggingFace Spaces serve content over HTTPS
• Robot daemon runs on http://localhost:8000 (HTTP, not HTTPS)
• Modern browsers block HTTPS → HTTP requests (mixed content policy)
• This blocks fetch() to the robot's REST API

Current Solution:

• WebSocket connections are allowed (less strict security)
• Use WebSocket /ws/set_target as fallback
• Limitation: No duration control via WebSocket

Why We Can't Fix This:

• Robot daemon is on user's local machine
• Space backend runs on HuggingFace servers (can't reach localhost)
• Can't proxy through Space backend
• Can't get robot's current state for client-side interpolation

Alternative Approaches (Not Implemented):

1. Client-side interpolation: Would need current robot state (blocked by same security issue)
2. Local HTTPS proxy: Too complex for end users
3. Run robot daemon on HTTPS: Requires SSL certificates for localhost

Deployment Gotchas

Binary Files:

• HuggingFace Spaces reject binary files (WAV, TTF, PNG, etc.)
• Must remove from reachy_mini/ and rmscript/ copies before pushing
• Already removed: docs/, examples/, assets/, fonts/

OpenGL Dependencies:

• OpenCV (cv2) requires system libraries: libgl1, libglib2.0-0, etc.
• Must install in Dockerfile (see lines 8-15)
• Package name changed from libgl1-mesa-glx → libgl1 in newer Debian

Git Submodules:

• Don't copy .git directories from rmscript/ and reachy_mini/
• Use rsync --exclude='.git' or remove after copying
• Otherwise Space treats them as submodules (causes issues)

Future Improvements

Potential Enhancements

1. Sound & Picture Support:

   • Currently IR supports these actions but execution skips them
   • Would need to implement in frontend/app.js:executeScript()

2. Syntax Highlighting:

   • Add CodeMirror or Monaco Editor for better editing experience
   • Custom rmscript syntax highlighting rules

3. Save/Load Scripts:

   • localStorage for saving scripts in browser
   • Or integrate with HuggingFace Datasets for cloud storage

4. Real-time Error Highlighting:

   • Show errors inline in editor (like VS Code)
   • Currently only shows in console panel

5. Movement Visualization:

   • 3D preview of robot movements before execution
   • Using Three.js or similar

6. Execution Controls:

   • Pause/resume execution
   • Step through script line-by-line
   • Stop/cancel running script

Known Issues

• Spaces timing limitation: Cannot be fixed without changing browser security or deployment architecture
• Cached IR bug: Fixed - now always compiles fresh editor content
• Euler angle conversion: Works but could add gimbal lock handling improvements

Development History & Fixes

Issues Fixed During Development:

1. Robot not executing moves (2025-11-26):

   • Problem: Robot daemon connected but moves didn't execute
   • Cause: Frontend was extracting translation but hardcoding euler angles to 0
   • Fix: Implemented matrixToEulerXYZ() to properly extract rotation from 4x4 matrices
   • Location: frontend/app.js:328-355

2. Modified scripts not executing (2025-11-26):

   • Problem: Editing script in editor didn't affect execution - same old script ran
   • Cause: executeScript() was using cached IR instead of compiling fresh
   • Fix: Always compile editor content before execution
   • Location: frontend/app.js:285-290

3. Local development broken after Space changes (2025-11-26):

   • Problem: Opening index.html locally didn't load app.js
   • Cause: Changed script path to /static/app.js for Space deployment
   • Fix: Auto-detect environment and load appropriate path
   • Location: frontend/index.html:320-329

4. Binary files rejected during Space deployment (2025-11-26):

   • Problem: HuggingFace push rejected due to WAV/TTF/PNG files
   • Cause: HuggingFace Spaces don't allow binary files without LFS
   • Fix: Remove docs, examples, assets from copied dependencies
   • Commands: See deployment section step 1

5. Docker build failing on Spaces (2025-11-26):

   • Problem: libgl1-mesa-glx package not found during build
   • Cause: Package renamed in newer Debian (Trixie)
   • Fix: Changed to libgl1 in Dockerfile
   • Location: Dockerfile:10

6. Duration not respected in execution (2025-11-26):

   • Problem: All moves happened instantly regardless of specified duration
   • Cause: Using WebSocket /ws/set_target which doesn't support duration
   • Fix: Switch to REST API POST /api/move/goto with duration parameter
   • Location: frontend/app.js:394-405

7. Fetch failures on HuggingFace Spaces (2025-11-26):

   • Problem: "Failed to fetch" errors when executing on Space
   • Cause: Browser mixed content policy blocks HTTPS → HTTP localhost
   • Fix: Fallback to WebSocket when fetch fails (timing limitation accepted)
   • Location: frontend/app.js:407-426

Troubleshooting

Backend won't start locally:

# Make sure rmscript is installed
cd ../rmscript && uv sync
cd ../rmscript_space_demo/backend

# Install backend deps
uv sync
uv run python app.py

Robot not connecting:

• Check robot daemon is running: curl http://localhost:8000/api/health
• Check WebSocket connection in browser console
• Try reloading the page

Space build fails:

• Check build logs in Space's "Build" tab
• Common issues: binary files not removed, missing system dependencies
• Dockerfile may need updating for new Debian versions

Compilation errors:

• Check rmscript syntax in examples
• Use "Verify" button before "Execute"
• Check console for detailed error messages

Credits

Built with:

• rmscript - Kid-friendly robot programming language
• reachy_mini - Reachy Mini SDK
• FastAPI - Backend framework
• HuggingFace Spaces - Deployment platform