README.md

---
title: RMScript Web Demo
emoji: 🤖
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
---

# RMScript Web Demo

Web-based IDE for writing, verifying, and executing rmscript code on Reachy Mini.

## Architecture

### Components

- **Frontend** (`frontend/`): Pure HTML/CSS/JavaScript web IDE
  - `index.html`: UI layout with embedded CSS styling
  - `app.js`: Application logic, compilation API calls, robot communication
  - Auto-detects environment (file:// for local, web server for production)

- **Backend** (`backend/`): FastAPI server (Python 3.11)
  - `app.py`: REST API for rmscript compilation and validation
  - Serves frontend files via static file mounting
  - Runs on port 8001 (local) or port 7860 (HuggingFace Spaces)

- **Robot Daemon**: Reachy Mini daemon (expected on localhost:8000)
  - REST API: `/api/move/goto` (with duration support)
  - WebSocket API: `/api/move/ws/set_target` (streaming, no duration)
  - State API: `/api/state/*` (robot current state)

### Communication Flow

1. **Script Verification/Compilation**:
   - Frontend → Backend (`/api/verify` or `/api/compile`)
   - Backend compiles rmscript using the rmscript package
   - Returns IR (Intermediate Representation) + errors/warnings

2. **Script Execution** (Local):
   - Frontend → Robot Daemon (`POST /api/move/goto`)
   - Each IR action sent with duration and interpolation mode
   - Proper timing with minimum-jerk interpolation

3. **Script Execution** (HuggingFace Spaces):
   - Frontend → Robot Daemon (WebSocket `/api/move/ws/set_target`)
   - Fallback due to HTTPS → HTTP localhost blocking
   - No duration control - moves are instant

## Quick Start

### On HuggingFace Spaces

Just open the Space - the web IDE will load automatically.

**Requirements:**
- Reachy Mini daemon running on `localhost:8000` on your local machine
- Modern web browser with WebSocket support

### Local Development

1. Install rmscript package:
```bash
cd rmscript
uv sync
```

2. Start backend server:
```bash
cd backend
uv sync
uv run python app.py
```

3. Make sure your Reachy Mini daemon is running on `localhost:8000`

4. Open `frontend/index.html` in your web browser

## Features

- **Code Editor**: Write rmscript with syntax examples
- **Verify Button**: Check script validity without executing
- **Execute Button**: Compile and run script on connected robot
- **Live Compilation**: See IR (Intermediate Representation) of your script
- **Example Scripts**: Load pre-written examples (basic, complex, repeat)
- **Real-time Logging**: Console shows compilation errors, warnings, and execution progress

## Usage

1. Write rmscript in the editor (or load an example)
2. Click "Verify" to check syntax and semantics
3. Review any errors/warnings in the console
4. Click "Execute" to run on the robot (robot must be connected)
5. Watch execution progress in the Execution Info panel

## Current Limitations

- Sound and picture actions are not yet implemented in execution
- **Movement timing on HuggingFace Spaces**: Due to browser security (HTTPS cannot fetch `http://localhost`), the Space version cannot use the robot's `/goto` endpoint which supports duration control.
  - **On Spaces**: Robot moves to each target as fast as possible (instant), but script execution pauses for the specified duration before the next move
  - **Locally**: Robot moves smoothly with proper duration and minimum-jerk interpolation
  - **Recommendation**: For demos requiring accurate movement timing, run locally

## Deployment to HuggingFace Spaces

To deploy this demo to HuggingFace Spaces:

1. **Prepare the repository structure:**
   - Copy the `rmscript` and `reachy_mini` directories (without `.git` and binary files):
     ```bash
     cd rmscript_space_demo

     # Option 1: Using rsync (recommended - cleaner)
     rsync -av --exclude='.git' --exclude='.venv' --exclude='__pycache__' \
           --exclude='*.pyc' ../rmscript/ ./rmscript/
     rsync -av --exclude='.git' --exclude='.venv' --exclude='__pycache__' \
           --exclude='*.pyc' ../reachy_mini/ ./reachy_mini/

     # Option 2: Using cp then cleanup
     cp -r ../rmscript ./rmscript
     cp -r ../reachy_mini ./reachy_mini
     rm -rf ./rmscript/.git ./reachy_mini/.git
     rm -rf ./rmscript/.venv ./reachy_mini/.venv

     # Remove binary files that HuggingFace Spaces rejects
     rm -rf ./reachy_mini/src/reachy_mini/assets/*.wav
     rm -rf ./reachy_mini/src/reachy_mini/daemon/app/dashboard/static/fonts/*.ttf
     rm -rf ./reachy_mini/docs ./reachy_mini/examples
     rm -rf ./reachy_mini/src/reachy_mini/descriptions/reachy_mini/mjcf
     rm -rf ./rmscript/examples ./rmscript/tests
     ```

   - Your structure should be:
     ```
     rmscript_space_demo/
       ├── Dockerfile
       ├── README.md
       ├── .dockerignore
       ├── backend/
       ├── frontend/
       ├── rmscript/     ← copied source (no .git, no binaries)
       └── reachy_mini/  ← copied source (no .git, no binaries)
     ```

2. **Create a new Space:**
   - Go to https://huggingface.co/new-space
   - Select "Docker" as the SDK
   - Push this directory to your Space repository

3. **The Space will:**
   - Build the Docker container
   - Install rmscript and dependencies
   - Serve the web IDE on port 7860

**Note:** The Space connects to your local robot daemon on `localhost:8000`. Make sure it's running when using the demo.

## Development

### File Structure

```
rmscript_space_demo/
├── Dockerfile              # Docker build config for HuggingFace Spaces
├── .dockerignore          # Excludes .git, .venv, __pycache__, etc.
├── README.md              # This file (includes HF Space metadata in YAML)
├── backend/
│   ├── app.py            # FastAPI server (compilation + static file serving)
│   └── pyproject.toml    # Python dependencies
└── frontend/
    ├── index.html        # Web IDE UI (with embedded CSS)
    └── app.js            # Frontend logic (compilation + execution)
```

### Key Implementation Details

**Matrix to Euler Conversion** (`frontend/app.js:328-355`):
- Converts 4x4 pose matrices (from rmscript IR) to euler angles (roll, pitch, yaw)
- Uses XYZ euler convention to match `scipy.spatial.transform.Rotation.as_euler("xyz")`
- Required because WebSocket API expects `{x, y, z, roll, pitch, yaw}` format

**Environment Detection** (`frontend/app.js:9-11`):
- Detects `file://` protocol → uses `http://localhost:8001` for backend
- Detects web server → uses relative URLs (same origin)
- Script loading path adapts similarly for `app.js` imports

**Execution Strategy** (`frontend/app.js:357-427`):
- **Tries** `POST /api/move/goto` with duration (works locally)
- **Catches** fetch failures → falls back to WebSocket (for Spaces)
- Logs warnings when fallback is used

### Modifying the Demo

**Frontend Changes:**
- Edit `frontend/app.js` for logic changes
- Edit `frontend/index.html` for UI/styling changes
- Test by opening `index.html` directly in browser (local mode)

**Backend Changes:**
- Edit `backend/app.py` for API changes
- Run locally: `cd backend && uv run python app.py`
- Test API at `http://localhost:8001/docs` (FastAPI Swagger UI)

**Deployment:**
- Changes auto-deploy when pushed to HuggingFace Space repo
- Build logs visible in Space's "Build" tab
- Runtime logs visible in Space's "Logs" tab

## Technical Notes

### Browser Security & Mixed Content

**The Problem:**
- HuggingFace Spaces serve content over HTTPS
- Robot daemon runs on `http://localhost:8000` (HTTP, not HTTPS)
- Modern browsers block HTTPS → HTTP requests (mixed content policy)
- This blocks `fetch()` to the robot's REST API

**Current Solution:**
- WebSocket connections are allowed (less strict security)
- Use WebSocket `/ws/set_target` as fallback
- Limitation: No duration control via WebSocket

**Why We Can't Fix This:**
- Robot daemon is on user's local machine
- Space backend runs on HuggingFace servers (can't reach localhost)
- Can't proxy through Space backend
- Can't get robot's current state for client-side interpolation

**Alternative Approaches (Not Implemented):**
1. **Client-side interpolation**: Would need current robot state (blocked by same security issue)
2. **Local HTTPS proxy**: Too complex for end users
3. **Run robot daemon on HTTPS**: Requires SSL certificates for localhost

### Deployment Gotchas

**Binary Files:**
- HuggingFace Spaces reject binary files (WAV, TTF, PNG, etc.)
- Must remove from `reachy_mini/` and `rmscript/` copies before pushing
- Already removed: docs/, examples/, assets/, fonts/

**OpenGL Dependencies:**
- OpenCV (cv2) requires system libraries: `libgl1`, `libglib2.0-0`, etc.
- Must install in Dockerfile (see lines 8-15)
- Package name changed from `libgl1-mesa-glx` → `libgl1` in newer Debian

**Git Submodules:**
- Don't copy `.git` directories from `rmscript/` and `reachy_mini/`
- Use `rsync --exclude='.git'` or remove after copying
- Otherwise Space treats them as submodules (causes issues)

## Future Improvements

### Potential Enhancements

1. **Sound & Picture Support:**
   - Currently IR supports these actions but execution skips them
   - Would need to implement in `frontend/app.js:executeScript()`

2. **Syntax Highlighting:**
   - Add CodeMirror or Monaco Editor for better editing experience
   - Custom rmscript syntax highlighting rules

3. **Save/Load Scripts:**
   - localStorage for saving scripts in browser
   - Or integrate with HuggingFace Datasets for cloud storage

4. **Real-time Error Highlighting:**
   - Show errors inline in editor (like VS Code)
   - Currently only shows in console panel

5. **Movement Visualization:**
   - 3D preview of robot movements before execution
   - Using Three.js or similar

6. **Execution Controls:**
   - Pause/resume execution
   - Step through script line-by-line
   - Stop/cancel running script

### Known Issues

- **Spaces timing limitation**: Cannot be fixed without changing browser security or deployment architecture
- **Cached IR bug**: Fixed - now always compiles fresh editor content
- **Euler angle conversion**: Works but could add gimbal lock handling improvements

### Development History & Fixes

**Issues Fixed During Development:**

1. **Robot not executing moves** (2025-11-26):
   - **Problem**: Robot daemon connected but moves didn't execute
   - **Cause**: Frontend was extracting translation but hardcoding euler angles to 0
   - **Fix**: Implemented `matrixToEulerXYZ()` to properly extract rotation from 4x4 matrices
   - **Location**: `frontend/app.js:328-355`

2. **Modified scripts not executing** (2025-11-26):
   - **Problem**: Editing script in editor didn't affect execution - same old script ran
   - **Cause**: `executeScript()` was using cached IR instead of compiling fresh
   - **Fix**: Always compile editor content before execution
   - **Location**: `frontend/app.js:285-290`

3. **Local development broken after Space changes** (2025-11-26):
   - **Problem**: Opening `index.html` locally didn't load `app.js`
   - **Cause**: Changed script path to `/static/app.js` for Space deployment
   - **Fix**: Auto-detect environment and load appropriate path
   - **Location**: `frontend/index.html:320-329`

4. **Binary files rejected during Space deployment** (2025-11-26):
   - **Problem**: HuggingFace push rejected due to WAV/TTF/PNG files
   - **Cause**: HuggingFace Spaces don't allow binary files without LFS
   - **Fix**: Remove docs, examples, assets from copied dependencies
   - **Commands**: See deployment section step 1

5. **Docker build failing on Spaces** (2025-11-26):
   - **Problem**: `libgl1-mesa-glx` package not found during build
   - **Cause**: Package renamed in newer Debian (Trixie)
   - **Fix**: Changed to `libgl1` in Dockerfile
   - **Location**: `Dockerfile:10`

6. **Duration not respected in execution** (2025-11-26):
   - **Problem**: All moves happened instantly regardless of specified duration
   - **Cause**: Using WebSocket `/ws/set_target` which doesn't support duration
   - **Fix**: Switch to REST API `POST /api/move/goto` with duration parameter
   - **Location**: `frontend/app.js:394-405`

7. **Fetch failures on HuggingFace Spaces** (2025-11-26):
   - **Problem**: "Failed to fetch" errors when executing on Space
   - **Cause**: Browser mixed content policy blocks HTTPS → HTTP localhost
   - **Fix**: Fallback to WebSocket when fetch fails (timing limitation accepted)
   - **Location**: `frontend/app.js:407-426`

## Troubleshooting

**Backend won't start locally:**
```bash
# Make sure rmscript is installed
cd ../rmscript && uv sync
cd ../rmscript_space_demo/backend

# Install backend deps
uv sync
uv run python app.py
```

**Robot not connecting:**
- Check robot daemon is running: `curl http://localhost:8000/api/health`
- Check WebSocket connection in browser console
- Try reloading the page

**Space build fails:**
- Check build logs in Space's "Build" tab
- Common issues: binary files not removed, missing system dependencies
- Dockerfile may need updating for new Debian versions

**Compilation errors:**
- Check rmscript syntax in examples
- Use "Verify" button before "Execute"
- Check console for detailed error messages

## Credits

Built with:
- [rmscript](../rmscript/) - Kid-friendly robot programming language
- [reachy_mini](../reachy_mini/) - Reachy Mini SDK
- [FastAPI](https://fastapi.tiangolo.com/) - Backend framework
- [HuggingFace Spaces](https://huggingface.co/spaces) - Deployment platform