changed gesture loader, it has tai chi and gangnam style now and the ability to load choregraphe gestures

README.md

@@ -0,0 +1,105 @@
+# naowalk
+
+A keyboard teleop client for the NAO robot: live video, walking, head control,
+two-way audio (hear the robot's mics, make it speak via TTS), and a YouTube
+music player that downloads a track with `yt-dlp` and plays it through the
+robot's own speakers.
+
+## Requirements
+
+**System packages** (Debian/Ubuntu names — adjust for your OS):
+
+```bash
+sudo apt install portaudio19-dev ffmpeg sshpass openssh-client
+```
+
+- `portaudio19-dev` — needed to build/install `pyaudio`
+- `ffmpeg` — needed by `yt-dlp` to extract audio to mp3
+- `sshpass` + `openssh-client` — used to `scp` downloaded tracks onto the
+  robot and to run `mkdir` over `ssh` (the script assumes the default
+  `nao`/`nao` credentials)
+
+**NAOqi Python SDK (`qi`)** — not on PyPI. Download the `pynaoqi-python`
+wheel that matches your OS/Python version from the Aldebaran/SoftBank
+Robotics developer site and install it:
+
+```bash
+pip install ./pynaoqi-*.whl
+```
+
+**Python packages:**
+
+```bash
+pip install -r requirements.txt
+```
+
+## Usage
+
+```bash
+python3 naowalk.py --ip <robot-ip> [options]
+```
+
+| Flag | Default | Description |
+|---|---|---|
+| `--ip` | `127.0.0.1` | Robot's IP address |
+| `--port` | `9559` | NAOqi port |
+| `--audio-device-index` | auto | Output device index for hearing the robot's mic on your machine |
+| `--mic-channel` | `left` | Which robot mic to stream (`left`, `right`, `front`, `rear`) — pick a working one, see Troubleshooting |
+| `--mic-gain` | `8.0` | Amplification applied to the streamed mic audio |
+| `--video-scale` | `2.0` | Display window scale (base feed is 320x240) |
+
+## Controls
+
+| Key | Action |
+|---|---|
+| `W`/`A`/`S`/`D` or arrows | Walk forward/turn/back |
+| `I`/`K`/`J`/`L` | Head pitch/yaw |
+| `Space` | Reset head to center |
+| `1` | Wave gesture |
+| `M` | Open music search (type query, `Enter` to search, `↑↓` to select, `Enter`/`1`-`5` to play) |
+| `P` | Pause / resume music (resume restarts the track from the beginning — see Notes) |
+| `X` | Stop music |
+| `-` / `=` | Volume down/up |
+| `T` | Type a message for the robot to speak (TTS) |
+| `Esc` | Quit (or cancel current mode) |
+
+## How it works
+
+- **`SoundReceiver`** — registered as a NAOqi service that receives the
+  robot's mic stream and plays it out your local speakers via `pyaudio`, with
+  gain and a prebuffer to smooth out network jitter.
+- **`NaoMusicPlayer`** — searches YouTube with `yt-dlp`, downloads and
+  converts the top match to mp3, `scp`s it to the robot, and plays it with
+  `ALAudioPlayer`.
+- **`NaoTeleop`** — the main loop: pygame window for video/input, keyboard
+  handling, head/walk control, and wiring everything above together.
+
+## Notes / design decisions
+
+- **No seek.** Seeking was removed entirely — it added the most complexity
+  for the least-used feature. `P` is a simple play/stop toggle; resuming
+  restarts the track from the beginning rather than tracking a playback
+  position.
+- **`moveInit()` on startup.** The robot used to need its walk buttons
+  jittered for a few seconds before its first walk would work. `run()` now
+  calls `ALMotionProxy::moveInit()` once at startup, which settles the walk
+  engine into the right pose before any real move command — this is the
+  documented NAOqi fix for exactly this symptom.
+- **`_safe()` helper.** Most one-off NAOqi calls (setting angles, volume,
+  stopping movement, unsubscribing services) are wrapped in a small
+  `self._safe(label, fn)` helper instead of a repeated `try/except` block, so
+  one failed call logs and moves on without crashing the teleop loop or the
+  shutdown sequence.
+
+## Troubleshooting
+
+- **Mic sounds dead / silent on one channel:** some robots have one or more
+  physical mics that produce no signal. Test all four with a quick energy
+  check, then pick a working channel with `--mic-channel`.
+- **Robot falls over on the very first walk command:** make sure you're on a
+  build that includes the `moveInit()` startup call described above; if it's
+  still happening, increase the `time.sleep(1.0)` after `goToPosture` to give
+  the posture more time to settle first.
+- **`scp`/`ssh` failures for music:** confirm the robot is reachable at
+  `--ip` and that `nao`/`nao` are still the correct credentials; the script
+  hardcodes them via `sshpass`.

chat-history/claude_automate-nao-video-server-deployment-and-cleanup_2026-07-21T22-15-10+0000.md

@@ -0,0 +1,66 @@
+---
+title: "Automate NAO video server deployment and cleanup"
+tags: []
+author: claude
+count: 2
+exporter: 3.1.0
+date: 2026-07-21T22-15-10+0000
+url: https://claude.ai/chat/44ac3c8f-f04d-4b23-a71d-f35c28b50a86
+---
+
+# Automate NAO video server deployment and cleanup
+
+## Table of Contents
+
+- [1: make naowalk automatically scp the video server to the nao's home dire…](#chat-1)
+- [2: lets gut the original audio and video code out, i want it all going th…](#chat-2)
+
+## chat-1
+
+> make naowalk automatically scp the video server to the nao's home directory, and run, then terminate the video server on close
+
+Now I'll add a launcher class that deploys and runs `nao_video_server.py` over SSH, then wire it into startup/shutdown.
+
+Now update the `NaoTeleop.__init__` signature and wire in the deploy call before video/audio init:
+
+Now add teardown in the shutdown block:
+
+Now add the CLI flags and pass them through:
+
+Let’s update the `--media-relay` help text, since it currently implies the server must already be running manually:
+
+Now let's sanity-check the file compiles and the diff looks right:
+
+Diff looks clean. Now let's copy the finished file to outputs and present it.
+
+___
+###### [top](#table-of-contents)
+## chat-2
+
+> lets gut the original audio and video code out, i want it all going through the relay, including my mic. Also lets see if we can improve the latency on the nao's mic audio
+
+Good, this stays unchanged (`start()`/`stop()` API preserved). Now let's check the HUD and shutdown/argparse sections:
+
+That's fine to leave as-is (harmless dead branch for "connecting"). Now the shutdown block and CLI/argparse section:
+
+Now let's do the bigger refactor. Starting with `nao_video_server.py` — adding the bidirectional talk channel and tightening audio buffering for latency:
+
+Given the scope of changes, I'll rewrite `nao_video_server.py` in full rather than patching piecemeal:
+
+Now let's do the equivalent large refactor on `naowalk.py`, section by section. First, the protocol constants and header comment:
+
+Now let's tune `SoundReceiver` for lower latency:
+
+Now update `feed()` to actively trim the backlog instead of just dropping one chunk on overflow:
+
+Now let's replace `LiveMicRelay` with the new `PushToTalk` class that talks over the media relay instead of SSH:
+
+Now I'll replace the entire `LiveMicRelay` class with `PushToTalk`:
+
+Now let's update `NaoTeleop.__init__` and the video/audio init methods:
+
+Now I'll replace this whole block with the simplified, relay-only version:
+
+___
+###### [top](#table-of-contents)
+

chat-history/claude_refactoring-python-project-files-and-code_2026-07-16T17-38-58+0000.md

@@ -0,0 +1,41 @@
+---
+title: "Refactoring Python project files and code"
+tags: []
+author: claude
+count: 2
+exporter: 3.1.0
+date: 2026-07-16T17-38-58+0000
+url: https://claude.ai/chat/313cfd88-e6b9-4256-b155-0b389d2b86c4
+---
+
+# Refactoring Python project files and code
+
+## Table of Contents
+
+- [1: make a requirements.txt, readme.md, and also remove the seek function…](#chat-1)
+- [2: how much faster can i make it walk, can we make it adjustable during r…](#chat-2)
+
+## chat-1
+
+> make a requirements.txt, readme.md, and also remove the seek function completely and reduce the lines of code without hurting stability
+
+Now the `pause()` method — simplify to a plain stop/restart toggle (no seek, no position math):
+
+Now let's add the `_safe` helper to `NaoTeleop` and use it to collapse the repeated try/except boilerplate:
+
+Now the shutdown block — this is where `_safe` pays off the most:
+
+Good — that's stable. Now let's view the final file once more end-to-end to make sure everything reads cleanly before shipping it.
+
+This looks clean and correct. Now let's write the `requirements.txt` and `README.md`.
+
+Good, all confirmed. Now the README:
+
+Now let's copy everything to outputs and present the files:
+
+___
+###### [top](#table-of-contents)
+## chat-2
+
+> how much faster can i make it walk, can we make it adjustable during runtime? we might need to change its posture so its less likely to fall and more balanced for carpet
+

chat-history/claude_replacing-music-player-backend-with-yt-dlp_2026-07-16T17-20-34+0000.md

@@ -0,0 +1,56 @@
+---
+title: "Replacing music player backend with yt-dlp"
+tags: []
+author: claude
+count: 2
+exporter: 3.1.0
+date: 2026-07-16T17-20-34+0000
+url: https://claude.ai/chat/8bb5e420-7ad2-4048-be3f-95894713aff6
+---
+
+# Replacing music player backend with yt-dlp
+
+## Table of Contents
+
+- [1: lets just gut the seek function and focus on what works. we also need…](#chat-1)
+- [2: alright awesome, lets remove the start from current position also, i w…](#chat-2)
+
+## chat-1
+
+> lets just gut the seek function and focus on what works. we also need to address the robot falling over if i try to walk from standstill, but only the first time i do it. If i jitter the walk forward and back buttons for about 3 seconds to get over the initial walk issues, it can start walking without falling over. how do we fix this
+
+Both very solvable. Let's do the walk fix first since it's the more important one — confirmed via NAOqi docs: `ALMotionProxy::moveInit()` is the standard fix for exactly this. It's a one-time blocking call that initializes the walk engine and settles the robot into the right pose _before_ the first real move command, which is exactly what your button-jittering was accidentally doing by brute force. It's missing from the script entirely.
+
+Now let's gut `seek()` entirely per your request — but keep the position tracking it shares with `pause()`, since that's still needed for resume-from-position to work:
+
+Right, orphaned body left over. Let's clean that up:
+
+Now remove the comma/period key bindings that called `seek()`:
+
+Now compile-check and confirm nothing else references `seek`:
+
+Clean. Let's ship it:
+
+___
+###### [top](#table-of-contents)
+## chat-2
+
+> alright awesome, lets remove the start from current position also, i want to get the lines of code down, what can be done without compromising on the stability and functionality to simplify our code
+
+Now let's do the full pass. First, the concrete simplification you asked for — remove position tracking, revert pause/resume to restart-from-beginning:
+
+Now `pause()` becomes a plain stop/restart toggle — no position math, no `playFileFromPosition`:
+
+Now let's cut the repetitive `try/except Exception as e: print(...)` boilerplate scattered through `NaoTeleop` — same safety, less code, via one small helper:
+
+Now let's add the `_safe` helper and apply it across these repeated sites:
+
+Now the `moveInit` startup call:
+
+Now the shutdown block — this is where it pays off most:
+
+Let's check the movement block and the `T` key's `stopMove()` call too:
+
+___
+###### [top](#table-of-contents)
+

chat-history/claude_volume-display-and-audio-underrun-fixes_2026-07-15T19-21-17+0000.md

@@ -0,0 +1,78 @@
+---
+title: "Volume display and audio underrun fixes"
+tags: []
+author: claude
+count: 2
+exporter: 3.1.0
+date: 2026-07-15T19-21-17+0000
+url: https://claude.ai/chat/0c44df5d-e4f0-425d-9730-53563dd8f762
+---
+
+# Volume display and audio underrun fixes
+
+## Table of Contents
+
+- [1: The wav file had distant sounds from my coworkers yapping. I didnt get…](#chat-1)
+- [2: audio works but is too quiet, amplify it and mix it all to make it mon…](#chat-2)
+
+## chat-1
+
+> The wav file had distant sounds from my coworkers yapping. I didnt get to talk but that proves the mic does do something
+> 
+> ┌──(lucka㉿theuniverseismine)-[~/programming/py/naowalk] 
+> └─$ python3 naomictest.py --ip spike.local --port 9561 
+> [W] 1784142061.737086 18238 qi.path.sdklayout: No Application was created, trying to deduce paths 
+> ✅ Connected to spike.local 
+> 
+> === TEST 1: Mic hardware energy (all 4 mics) === 
+> Make noise near the robot's head (clap, talk right up close to it) for 10s... 
+>  t=0s  front=1.86  rear=1.76  left=22.41  right=25.90 
+>  t=1s  front=1.78  rear=1.66  left=10.84  right=12.57 
+>  t=2s  front=1.77  rear=1.75  left=12.23  right=13.08 
+>  t=3s  front=1.75  rear=1.73  left=18.84  right=19.08 
+>  t=4s  front=1.69  rear=1.78  left=18.88  right=18.45 
+>  t=5s  front=1.73  rear=1.75  left=17.76  right=19.11 
+>  t=6s  front=1.71  rear=1.79  left=15.86  right=14.67 
+>  t=7s  front=1.73  rear=1.84  left=20.97  right=21.05 
+>  t=8s  front=1.83  rear=1.72  left=14.95  right=15.82 
+>  t=9s  front=1.68  rear=1.78  left=10.71  right=12.04 
+> ⚠ Flat mic(s): front, rear. The others responded normally, which points to a fault specific to front, rear rather than the audio 
+> pipeline as a whole (that pipeline is clearly working for the others). 
+> 
+> === TEST 2: Local on-robot recording === 
+> Recording 5s directly to /home/nao/mic_test.wav on the robot itself (this bypasses the network stream and your laptop's audio sta
+> ck entirely)... 
+> Done. Pull it off and play it with: 
+>  scp nao@spike.local:/home/nao/mic_test.wav . 
+>  (then open mic_test.wav in any media player - it's 4-channel, so check all channels) 
+> If that file has clearly audible sound, the mic and robot are fine and the 
+> bug is specifically in naowalk.py's live-streaming path. 
+> 
+> === TEST 3: Robot's own speaker === 
+> Press Enter to have the robot say a test phrase out loud...hi 
+> If you heard the robot speak just now, its speaker output is fine (this is unrelated to hearing its mic on your laptop, but rules
+> out a robot-wide audio fault). 
+> 
+> Done. Report back which test(s) failed and we'll know exactly where to look.
+
+That's the smoking gun. Look at the pattern: **left and right respond strongly to noise (10–25), front and rear are dead flat (~1.7–1.9)** — every single test agrees. And the recording that had audible coworker chatter makes total sense now: it's a 4-channel file, so the left/right tracks caught that audio while front/rear stayed silent.
+
+This was never a bug in the code. `naowalk.py` was explicitly requesting the **front** mic (`setClientPreferences(..., 3, 0)` — channel 3 = FRONTCHANNEL), which happens to be one of the two dead mics on this robot. It was faithfully streaming from a mic that produces no signal.
+
+Fix: point the stream at a working mic instead — left or right. Let me make it configurable so you can pick whichever sounds better:
+
+Now add the CLI flag and pass it through:
+
+Let's verify and ship it:
+
+___
+###### [top](#table-of-contents)
+## chat-2
+
+> audio works but is too quiet, amplify it and mix it all to make it mono
+
+Now let's rewrite `SoundReceiver` to request all mics, mix down to mono skipping dead channels, and apply amplification with a soft limiter:
+
+___
+###### [top](#table-of-contents)
+

chat-history/claude_your-first-chat-with-claude_2026-07-16T20-40-30+0000.md

@@ -0,0 +1,53 @@
+---
+title: "Your first chat with Claude"
+tags: []
+author: claude
+count: 5
+exporter: 3.1.0
+date: 2026-07-16T20-40-30+0000
+url: https://claude.ai/chat/838b001f-8bf8-4539-98ec-467513fb9860
+---
+
+# Your first chat with Claude
+
+## Table of Contents
+
+- [1: add the 6 7 meme as a third gesture](#chat-1)
+- [2: kinda looks like hes flailing his arms around, isnt it just the elbows…](#chat-2)
+- [3: right motions, but the nao currently extends his arms out, then moves…](#chat-3)
+- [4: perfect, now just rotate the hands so the palm faces up, everything el…](#chat-4)
+- [5: make him tts say six seven, also remove the input delay on starting th…](#chat-5)
+
+## chat-1
+
+> add the 6 7 meme as a third gesture
+
+## chat-2
+
+> kinda looks like hes flailing his arms around, isnt it just the elbows moving in the meme? with the arms in front
+
+Now let me replace the whole method with a version where the shoulders lock the arms up in front and only the elbows do the seesaw motion.
+
+___
+###### [top](#table-of-contents)
+## chat-3
+
+> right motions, but the nao currently extends his arms out, then moves his elbows to push the forearms close to his chest. i think the rotation is wrong on the elbow
+
+That makes sense — at `ElbowYaw=0` the elbow's bend axis is oriented so flexing curls the forearm in across the body toward the chest, not up/down in place. Rotating `ElbowYaw` about 90° turns that same hinge into a front-facing "tip" instead, and I'll shrink the swing so it's a wobble, not a full extend-then-fold.
+
+___
+###### [top](#table-of-contents)
+## chat-4
+
+> perfect, now just rotate the hands so the palm faces up, everything else is perfect
+
+## chat-5
+
+> make him tts say six seven, also remove the input delay on starting the 3rd gesture
+
+`self.tts.say(...)` is a blocking call, so it was speaking first and only starting the arm motion once the phrase finished — that's the delay. Making it fire on its own thread lets the gesture start immediately while the speech plays alongside it.
+
+___
+###### [top](#table-of-contents)
+

chat-history/scripts-used.txt

@@ -1 +1,2 @@
 https://github.com/iikoshteruu/enhanced-grok-export
+https://github.com/revivalstack/ai-chat-exporter

gestures/1_wave.py

@@ -0,0 +1,20 @@
+# Loaded by naowalk.py's gesture loader (_load_gestures). The filename's
+# leading digit ("1") is which number key triggers this. Every gesture
+# file just needs a run(motion, tts) function - motion is the connected
+# ALMotion proxy, tts is ALTextToSpeech. Optionally set NAME below to
+# control the display name shown in the on-screen controls hint.
+
+NAME = "Wave"
+
+
+def run(motion, tts):
+    motion.setStiffnesses("RArm", 1.0)
+    names = ["RShoulderPitch", "RShoulderRoll", "RElbowYaw", "RElbowRoll", "RWristYaw", "RHand"]
+    angles = [[-1.5]*5, [-0.4,-0.7,-0.4,-0.7,-0.4], [0.6,1.1,0.6,1.1,0.6],
+              [0.8,0.3,0.8,0.3,0.8], [0.0,1.5,0.0,-1.5,0.0], [1.0]*5]
+    times = [[0.5,1.0,1.5,2.0,2.5]] * 6
+    motion.angleInterpolation(names, angles, times, True)
+    neutral = [1.5, -0.15, 1.2, 0.5, 0.0, 0.0]
+    motion.angleInterpolationWithSpeed(names, neutral, 0.3)
+    # Relaxed stiffness after completion
+    motion.setStiffnesses("RArm", 0.3)

gestures/2_cena.py

@@ -0,0 +1,23 @@
+NAME = "Cena"
+
+
+def run(motion, tts):
+    tts.say("You can't see me")
+
+    motion.setStiffnesses("RArm", 1.0)
+    names = ["RShoulderPitch", "RShoulderRoll", "RElbowYaw", "RElbowRoll", "RWristYaw", "RHand"]
+
+    # Setup for tight, face-level sweep
+    tight_setup = [-0.4, -0.2, 0.0, 1.2, 1.5, 1.0]
+    motion.angleInterpolation(names, tight_setup, [0.5]*6, True)
+
+    # The horizontal "backhand" sweep
+    motion.angleInterpolation(["RElbowYaw"], [0.8], 0.4, True)
+    motion.angleInterpolation(["RElbowYaw"], [-0.8], 0.6, True)
+    motion.angleInterpolation(["RElbowYaw"], [0.0], 0.4, True)
+
+    # Return to neutral
+    neutral = [1.5, -0.15, 1.2, 0.5, 0.0, 0.0]
+    motion.angleInterpolationWithSpeed(names, neutral, 0.3)
+    # Relaxed stiffness after completion
+    motion.setStiffnesses("RArm", 0.3)

gestures/3_six_seven.py

@@ -0,0 +1,47 @@
+import threading
+
+NAME = "6-7"
+
+
+def run(motion, tts):
+    # Threaded so the TTS call doesn't block the arm motion below.
+    threading.Thread(target=tts.say, args=("six seven",), daemon=True).start()
+
+    motion.setStiffnesses(["RArm", "LArm"], 1.0)
+
+    # Lock both arms up in front of the body, hands held out like a scale.
+    # ElbowYaw is rotated ~90° off zero here - that's what points the elbow's
+    # bend axis so flexing tips the forearm forward/down instead of curling
+    # it in across the chest. These joints don't move again until the return.
+    hold_names = [
+        "RShoulderPitch", "RShoulderRoll", "RElbowYaw", "RWristYaw", "RHand",
+        "LShoulderPitch", "LShoulderRoll", "LElbowYaw", "LWristYaw", "LHand"
+    ]
+    hold_angles = [
+        0.2, -0.2, 1.4, 1.57, 0.6,
+        0.2, 0.2, -1.4, -1.57, 0.6
+    ]
+    motion.angleInterpolationWithSpeed(hold_names, hold_angles, 0.25)
+
+    # Only the elbows seesaw now - a small tip back and forth, one arm
+    # rising while the other falls, like weighing something in each hand
+    # ("six... seven..."). Small amplitude so it stays a wobble, not a fold.
+    elbow_names = ["RElbowRoll", "LElbowRoll"]
+    r_low, r_high = 0.6, 1.0
+    l_low, l_high = -0.6, -1.0
+
+    elbow_angles = [
+        [r_low, r_high, r_low, r_high, r_low, r_high],
+        [l_high, l_low, l_high, l_low, l_high, l_low],
+    ]
+    times = [[0.35, 0.7, 1.05, 1.4, 1.75, 2.1]] * 2
+    motion.angleInterpolation(elbow_names, elbow_angles, times, True)
+
+    # Return everything to neutral
+    neutral_names = hold_names + elbow_names
+    neutral = [1.5, -0.15, 1.2, 0.0, 0.0,
+               1.5, 0.15, -1.2, 0.0, 0.0,
+               0.5, -0.5]
+    motion.angleInterpolationWithSpeed(neutral_names, neutral, 0.3)
+    # Relaxed stiffness after completion
+    motion.setStiffnesses(["RArm", "LArm"], 0.3)

gestures/4_gangnam_style.py

@@ -0,0 +1,22 @@
+import os
+
+NAME = "Gangnam Style"
+
+# The Choregraphe project lives in a sibling folder with the same name
+# as this file (minus the .py) - e.g. gestures/4_gangnam_style/ next
+# to gestures/4_gangnam_style.py - containing manifest.xml and a
+# GangnamStyle/behavior.xar subfolder, exactly what Choregraphe's
+# "Export as package" produces.
+PACKAGE_DIR = os.path.splitext(os.path.abspath(__file__))[0]
+BEHAVIOR_SUBDIR = "GangnamStyle"
+
+
+def run(motion, tts, teleop):
+    # Deploys (first run only, cached after) + runs the actual
+    # Choregraphe behavior on the robot - this blocks until it
+    # finishes, same as the old hand-coded version did.
+    teleop.behavior_runner.ensure_running(PACKAGE_DIR, BEHAVIOR_SUBDIR)
+
+    # The .xar moves legs/hips, unlike the old arms-only version, so
+    # bring him back to a known walk-ready stance afterward.
+    teleop.idle_crouch()

gestures/4_gangnam_style/manifest.xml

@@ -0,0 +1,16 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<package uuid=".lastUploadedChoregrapheBehavior" version="0.0.0" installOnlyAtStartup="false">
+  <names>
+    <name lang="en_US">.lastUploadedChoregrapheBehavior</name>
+    <name lang="en_US-x-scrald">.lastUploadedChoregrapheBehavior</name>
+  </names>
+  <descriptionLanguages>
+    <language>en_US</language>
+  </descriptionLanguages>
+  <contents>
+    <behaviorContent path="GangnamStyle">
+      <nature>interactive</nature>
+      <permissions/>
+    </behaviorContent>
+  </contents>
+</package>

gestures/5_tai_chi.py

@@ -0,0 +1,22 @@
+import os
+
+NAME = "Tai Chi"
+
+# The Choregraphe project lives in a sibling folder with the same name
+# as this file (minus the .py) - e.g. gestures/5_tai_chi/ next to
+# gestures/5_tai_chi.py - containing manifest.xml and a
+# behavior_1/behavior.xar subfolder, exactly what Choregraphe's
+# "Export as package" produces.
+PACKAGE_DIR = os.path.splitext(os.path.abspath(__file__))[0]
+BEHAVIOR_SUBDIR = "behavior_1"
+
+
+def run(motion, tts, teleop):
+    # Deploys (first run only, cached after) + runs the actual
+    # Choregraphe behavior on the robot - this blocks until it
+    # finishes, same pattern as the Gangnam Style gesture.
+    teleop.behavior_runner.ensure_running(PACKAGE_DIR, BEHAVIOR_SUBDIR)
+
+    # Tai chi moves legs/hips through slow stances, so bring him back
+    # to a known walk-ready stance afterward.
+    teleop.idle_crouch()

gestures/5_tai_chi/5_tai_chi.pml

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<Package name="5_tai_chi" format_version="5">
+    <Manifest src="manifest.xml" />
+    <BehaviorDescriptions>
+        <BehaviorDescription name="behavior" src="tai_chi" xar="behavior.xar" />
+    </BehaviorDescriptions>
+    <Dialogs />
+    <Resources />
+    <Topics />
+    <IgnoredPaths />
+    <Translations auto-fill="en_US">
+        <Translation name="translation_en_US" src="translations/translation_en_US.ts" language="en_US" />
+    </Translations>
+</Package>

gestures/5_tai_chi/manifest.xml

@@ -0,0 +1,18 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<package version="0.0.0" uuid="5_tai_chi-0cc7f3" installOnlyAtStartup="false">
+ <names>
+  <name lang="en_US">Untitled</name>
+ </names>
+ <supportedLanguages>
+  <language>en_US</language>
+ </supportedLanguages>
+ <descriptionLanguages>
+  <language>en_US</language>
+ </descriptionLanguages>
+ <contents>
+  <behaviorContent path="tai_chi">
+   <nature>interactive</nature>
+   <permissions/>
+  </behaviorContent>
+ </contents>
+</package>

gestures/5_tai_chi/tai_chi/behavior.xar

@@ -0,0 +1,857 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<ChoregrapheProject xmlns="http://www.ald.softbankrobotics.com/schema/choregraphe/project.xsd" xar_version="3">
+  <Box name="root" id="-1" localization="8" tooltip="Root box of Choregraphe&apos;s behavior. Highest level possible." x="0" y="0">
+    <bitmap>media/images/box/root.png</bitmap>
+    <script language="4">
+      <content>
+        <![CDATA[]]>
+      </content>
+    </script>
+    <Input name="onLoad" type="1" type_size="1" nature="0" inner="1" tooltip="Signal sent when diagram is loaded." id="1" />
+    <Input name="onStart" type="1" type_size="1" nature="2" inner="0" tooltip="Box behavior starts when a signal is received on this input." id="2" />
+    <Input name="onStop" type="1" type_size="1" nature="3" inner="0" tooltip="Box behavior stops when a signal is received on this input." id="3" />
+    <Output name="onStopped" type="1" type_size="1" nature="1" inner="0" tooltip="Signal sent when box behavior is finished." id="4" />
+    <Timeline enable="0">
+      <BehaviorLayer name="behavior_layer1">
+        <BehaviorKeyframe name="keyframe1" index="1">
+          <Diagram>
+            <Box name="Tai Chi Chuan" id="11" localization="8" tooltip="Arms part of Tai Chi Chuan dance." x="198" y="164">
+              <bitmap>media/images/box/movement/dance.png</bitmap>
+              <script language="4">
+                <content>
+                  <![CDATA[]]>
+                </content>
+              </script>
+              <Input name="onLoad" type="1" type_size="1" nature="0" inner="1" tooltip="Signal sent when diagram is loaded." id="1" />
+              <Input name="onStart" type="1" type_size="1" nature="2" inner="0" tooltip="Box behavior starts when a signal is received on this input." id="2" />
+              <Input name="onStop" type="1" type_size="1" nature="3" inner="0" tooltip="Box behavior stops when a signal is received on this input." id="3" />
+              <Output name="onStopped" type="1" type_size="1" nature="1" inner="0" tooltip="Signal sent when box behavior is finished." id="4" />
+              <Timeline enable="1" fps="5" start_frame="1" end_frame="-1" size="250">
+                <BehaviorLayer name="LED">
+                  <BehaviorKeyframe name="keyframe15" index="15">
+                    <Diagram>
+                      <Box name="Light_ConcentratedEyes" id="1" localization="8" tooltip="Set an animated gaze which expresses concentration." x="231" y="105">
+                        <bitmap>media/images/box/interaction/LED.png</bitmap>
+                        <script language="4">
+                          <content>
+                            <![CDATA[class MyClass(GeneratedClass):
+  def __init__(self):
+    try: # disable autoBind;
+      GeneratedClass.__init__(self, False);
+    except TypeError: # if NAOqi < 1.14;
+      GeneratedClass.__init__( self );
+
+  def onLoad(self):
+    self.bIsRunning = False;
+    self.leds = self.session().service("ALLeds")
+
+  def onUnload(self):
+    self.onInput_onStop(); # will stop current loop execution
+
+  def onInput_onStart(self):
+    #self.log( self.getName() + ": start - begin" );
+
+    if( self.bIsRunning ):
+      #print( self.getName() + ": already started => nothing" );
+      return;
+
+    self.bIsRunning = True;
+    self.bMustStop = False;
+
+    rDuration = 1.0;
+    self.leds.fadeRGB( "FaceLedsTop", 0x0000ff, rDuration, _async=True );
+    self.leds.fadeRGB( "FaceLedsInternal", 0x00ff00, rDuration, _async=True );
+    self.leds.fadeRGB( "FaceLedsBottom", 0x0000ff, rDuration, _async=True );
+    self.leds.fadeRGB( "FaceLedsExternal", 0x0000ff, rDuration );
+
+    while( not self.bMustStop ):
+      rTime=1.0
+      self.leds.fadeRGB( "FaceLedsInternal", 0x000000, rTime );
+      if( self.bMustStop ):
+        break;
+      self.leds.fadeRGB( "FaceLedsInternal", 0x00ff00, rDuration );
+
+
+    # end while
+    self.bIsRunning = False;
+    self.onStopped();
+
+  def onInput_onStop(self):
+    self.bMustStop = True; # will stop current loop execution]]>
+                          </content>
+                        </script>
+                        <Input name="onLoad" type="1" type_size="1" nature="0" inner="1" tooltip="Signal sent when diagram is loaded." id="1" />
+                        <Input name="onStart" type="1" type_size="1" nature="2" inner="0" tooltip="Box behavior starts when a signal is received on this input." id="2" />
+                        <Input name="onStop" type="1" type_size="1" nature="3" inner="0" tooltip="Box behavior stops when a signal is received on this input." id="3" />
+                        <Output name="onStopped" type="1" type_size="1" nature="1" inner="0" tooltip="Signal sent when box behavior is finished." id="4" />
+                      </Box>
+                      <Box name="Tocolor" id="2" localization="8" tooltip="tags:&#x0A;LED" x="490" y="183">
+                        <bitmap>media/images/box/interaction/LED.png</bitmap>
+                        <script language="4">
+                          <content>
+                            <![CDATA[class MyClass(GeneratedClass):
+    def __init__(self):
+        GeneratedClass.__init__(self)
+
+    def onLoad(self):
+        #~ puts code for box initialization here
+        self.leds = self.session().service("ALLeds")
+
+    def onUnload(self):
+        #~ puts code for box cleanup here
+        self.leds.reset("EarLeds")
+        pass
+
+    def onInput_onStart(self):
+        rDuration = 0.2
+        self.leds.fadeRGB( "BrainLeds", 0xffffff, rDuration)
+
+        self.onStopped()
+
+    def onInput_onStop(self):
+        self.onUnload() #~ it is usually a good idea to call onUnload of this box in a onStop method, as the code written in onUnload is used to finish the working of the box as well
+        pass]]>
+                          </content>
+                        </script>
+                        <Input name="onLoad" type="1" type_size="1" nature="0" inner="1" tooltip="Signal sent when diagram is loaded." id="1" />
+                        <Input name="onStart" type="1" type_size="1" nature="2" inner="0" tooltip="Box behavior starts when a signal is received on this input." id="2" />
+                        <Input name="onStop" type="1" type_size="1" nature="3" inner="0" tooltip="Box behavior stops when a signal is received on this input." id="3" />
+                        <Output name="onStopped" type="1" type_size="1" nature="1" inner="0" tooltip="Signal sent when box behavior is finished." id="4" />
+                        <Resource name="Left eye leds" type="Lock" timeout="1" />
+                        <Resource name="Right eye leds" type="Lock" timeout="1" />
+                      </Box>
+                      <Link inputowner="1" indexofinput="2" outputowner="0" indexofoutput="1" />
+                    </Diagram>
+                  </BehaviorKeyframe>
+                  <BehaviorKeyframe name="Eye48" index="247">
+                    <Diagram>
+                      <Box name="Blink" id="3" localization="8" tooltip="tags:&#x0A;LED" x="339" y="161">
+                        <bitmap>media/images/box/interaction/LED.png</bitmap>
+                        <script language="4">
+                          <content>
+                            <![CDATA[class MyClass(GeneratedClass):
+    def __init__(self):
+        GeneratedClass.__init__(self)
+
+    def onLoad(self):
+        #~ puts code for box initialization here
+        self.leds = self.session().service("ALLeds")
+
+    def onUnload(self):
+        #~ puts code for box cleanup here
+        pass
+
+    def onInput_onStart(self):
+        rDuration = 0.05;
+        self.leds.fadeRGB( "FaceLed0", 0x000000, rDuration, _async=True );
+        self.leds.fadeRGB( "FaceLed1", 0x000000, rDuration, _async=True );
+        self.leds.fadeRGB( "FaceLed2", 0xffffff, rDuration, _async=True  );
+        self.leds.fadeRGB( "FaceLed3", 0x000000, rDuration, _async=True  );
+        self.leds.fadeRGB( "FaceLed4", 0x000000, rDuration, _async=True  );
+        self.leds.fadeRGB( "FaceLed5", 0x000000, rDuration, _async=True  );
+        self.leds.fadeRGB( "FaceLed6", 0xffffff, rDuration, _async=True  );
+        self.leds.fadeRGB( "FaceLed7", 0x000000, rDuration );
+
+        time.sleep( 0.1 );
+
+        rDuration = 0.05;
+        self.leds.fadeRGB( "FaceLeds", 0xffffff, rDuration );
+
+        self.onStopped()
+
+
+    def onInput_onStop(self):
+        self.onUnload() #~ it is usually a good idea to call onUnload of this box in a onStop method, as the code written in onUnload is used to finish the working of the box as well
+        pass]]>
+                          </content>
+                        </script>
+                        <Input name="onLoad" type="1" type_size="1" nature="0" inner="1" tooltip="Signal sent when diagram is loaded." id="1" />
+                        <Input name="onStart" type="1" type_size="1" nature="2" inner="0" tooltip="Box behavior starts when a signal is received on this input." id="2" />
+                        <Input name="onStop" type="1" type_size="1" nature="3" inner="0" tooltip="Box behavior stops when a signal is received on this input." id="3" />
+                        <Output name="onStopped" type="1" type_size="1" nature="1" inner="0" tooltip="Signal sent when box behavior is finished." id="4" />
+                        <Resource name="Left eye leds" type="Lock" timeout="1" />
+                        <Resource name="Right eye leds" type="Lock" timeout="1" />
+                      </Box>
+                      <Link inputowner="3" indexofinput="2" outputowner="0" indexofoutput="1" />
+                    </Diagram>
+                  </BehaviorKeyframe>
+                </BehaviorLayer>
+                <BehaviorLayer name="music">
+                  <BehaviorKeyframe name="keyframe12" index="12">
+                    <Diagram>
+                      <Box name="Play Sound" id="1" localization="8" tooltip="Play a sound file. Select the file in parameters. The format of the file can be wav or ogg (on virtual robot) and also mp3 (on a real robot).&#x0A;&#x0A;Note: There can be some delay to play ogg or mp3 (on a robot) files. We advise you to use wav if&#x0A;you want a fast play of the file." x="193" y="90">
+                        <bitmap>media/images/box/interaction/play_music.png</bitmap>
+                        <script language="4">
+                          <content>
+                            <![CDATA[class MyClass(GeneratedClass):
+    def __init__(self):
+        GeneratedClass.__init__(self, False)
+
+    def onLoad(self):
+        self.bIsRunning = False
+
+    def onUnload(self):
+        self.bIsRunning = False
+
+    def onInput_onStart(self):
+        self.bIsRunning = True
+
+    def onInput_onStop(self):
+        if( self.bIsRunning ):
+            self.onUnload()
+            self.onStopped()]]>
+                          </content>
+                        </script>
+                        <Input name="onLoad" type="1" type_size="1" nature="0" inner="1" tooltip="Signal sent when Diagram is loaded." id="1" />
+                        <Input name="onStart" type="1" type_size="1" nature="2" inner="0" tooltip="Starts the music." id="2" />
+                        <Input name="onStop" type="1" type_size="1" nature="3" inner="0" tooltip="Stops the music." id="3" />
+                        <Output name="onStopped" type="1" type_size="1" nature="1" inner="0" tooltip="Signal sent when box behavior is finished or stopped." id="4" />
+                        <Parameter name="File name" inherits_from_parent="0" content_type="4" value="/sounds/mikhael-landscape-paisaje.ogg" default_value="" tooltip="Name of the file you want to play.&#x0A;&#x0A;Note: You can click on the folder icon to browse the project content or import&#x0A;new files to the project." id="5" />
+                        <Parameter name="Begin position (s)" inherits_from_parent="0" content_type="2" value="0" default_value="0" min="0" max="600" tooltip="Position in seconds where the playing must start." id="6" />
+                        <Parameter name="Volume (%)" inherits_from_parent="0" content_type="1" value="100" default_value="100" min="0" max="100" tooltip="Volume the file is played with." id="7" />
+                        <Parameter name="Balance L/R" inherits_from_parent="0" content_type="2" value="0" default_value="0" min="-1" max="1" tooltip="Value which determines if the sound is played more on the robot&apos;s left or right.&#x0A;You can particularly set it to:&#x0A;- -1 to play only on the left loudspeaker.&#x0A;- 0 to play on both loudspeakers.&#x0A;- 1 to play only on the right loudspeaker." id="8" />
+                        <Parameter name="Play in loop" inherits_from_parent="0" content_type="0" value="0" default_value="0" tooltip="This parameter allows to play the file in loop. The playing will start each time at&#x0A;the beginning of the file." id="9" />
+                        <Timeline enable="0">
+                          <BehaviorLayer name="behavior_layer1">
+                            <BehaviorKeyframe name="keyframe1" index="1">
+                              <Diagram>
+                                <Box name="Play Sound File" id="2" localization="8" tooltip="Play the sound." x="442" y="70">
+                                  <bitmap>media/images/box/interaction/play_music.png</bitmap>
+                                  <script language="4">
+                                    <content>
+                                      <![CDATA[import time
+
+class MyClass(GeneratedClass):
+    def __init__(self):
+        GeneratedClass.__init__(self, False)
+
+    def onLoad(self):
+        self.player = self.session().service('ALAudioPlayer')
+        self.playerStop = self.session().service('ALAudioPlayer') #Create another service as wait is blocking if audioout is remote
+        self.bIsRunning = False
+        self.ids = []
+
+    def onUnload(self):
+        for id in self.ids:
+            try:
+                self.playerStop.stop(id)
+            except:
+                pass
+        while( self.bIsRunning ):
+            time.sleep( 0.2 )
+
+    def onInput_onStart(self, p):
+        self.bIsRunning = True
+        try:
+            if (self.getParameter("Play in loop")) :
+               id = self.player.pCall("playFileInLoop",p,self.getParameter("Volume (%)")/100.,self.getParameter("Balance L/R"))
+            else :
+               id = self.player.pCall("playFileFromPosition",p,self.getParameter("Begin position (s)"),self.getParameter("Volume (%)")/100.,self.getParameter("Balance L/R"))
+            self.ids.append(id)
+            self.player.wait(id)
+        finally:
+            try:
+                self.ids.remove(id)
+            except:
+                pass
+            if( self.ids == [] ):
+                self.onStopped() # activate output of the box
+                self.bIsRunning = False
+
+    def onInput_onStop(self):
+        self.onUnload()]]>
+                                    </content>
+                                  </script>
+                                  <Input name="onLoad" type="1" type_size="1" nature="0" inner="1" tooltip="Signal sent when Diagram is loaded." id="1" />
+                                  <Input name="onStart" type="3" type_size="1" nature="2" inner="0" tooltip="Box behavior starts when a signal is received on this Input." id="2" />
+                                  <Input name="onStop" type="1" type_size="1" nature="3" inner="0" tooltip="Box behavior stops when a signal is received on this Input." id="3" />
+                                  <Output name="onStopped" type="1" type_size="1" nature="1" inner="0" tooltip="Signal sent when Box behavior is finished." id="4" />
+                                  <Parameter name="Begin position (s)" inherits_from_parent="1" content_type="2" value="0" default_value="0" min="0" max="600" tooltip="Position in seconds where the playing must start." id="5" />
+                                  <Parameter name="Volume (%)" inherits_from_parent="1" content_type="1" value="100" default_value="100" min="0" max="100" tooltip="Volume the file is played with." id="6" />
+                                  <Parameter name="Balance L/R" inherits_from_parent="1" content_type="2" value="0" default_value="0" min="-1" max="1" tooltip="Value which determines if the sound is played more on the robot&apos;s left or right.&#x0A;You can particularly set it to:&#x0A;- -1 to play only on the left loudspeaker.&#x0A;- 0 to play on both loudspeakers.&#x0A;- 1 to play only on the right loudspeaker." id="7" />
+                                  <Parameter name="Play in loop" inherits_from_parent="1" content_type="0" value="0" default_value="0" tooltip="This parameter allows to play the file in loop. The playing will start each time at&#x0A;the beginning of the file." id="8" />
+                                </Box>
+                                <Box name="Get Attached File" id="1" localization="8" tooltip="Use this box to choose an attached file in its parameters. The filename will be sent on&#x0A;the output when the input is stimulated." x="216" y="71">
+                                  <bitmap>media/images/box/folder.png</bitmap>
+                                  <script language="4">
+                                    <content>
+                                      <![CDATA[class MyClass(GeneratedClass):
+    def __init__(self):
+        GeneratedClass.__init__(self, False)
+
+    def onLoad(self):
+        pass
+
+    def onInput_onStart(self):
+        self.onStopped(self.behaviorAbsolutePath() + self.getParameter("File name"))]]>
+                                    </content>
+                                  </script>
+                                  <Input name="onLoad" type="1" type_size="1" nature="0" inner="1" tooltip="Signal sent when diagram is loaded." id="1" />
+                                  <Input name="onStart" type="1" type_size="1" nature="1" inner="0" tooltip="To send the filepath on the output." id="2" />
+                                  <Output name="onStopped" type="3" type_size="1" nature="2" inner="0" tooltip="The filepath of the selected resource file." id="3" />
+                                  <Parameter name="File name" inherits_from_parent="1" content_type="4" value="" default_value="" tooltip="Name of the file which is going to be sent on the box output.&#x0A;&#x0A;Note: You can click on the folder icon to browse the project content or import&#x0A;new files to the project." id="4" />
+                                </Box>
+                                <Link inputowner="0" indexofinput="4" outputowner="2" indexofoutput="4" />
+                                <Link inputowner="1" indexofinput="2" outputowner="0" indexofoutput="2" />
+                                <Link inputowner="2" indexofinput="2" outputowner="1" indexofoutput="3" />
+                              </Diagram>
+                            </BehaviorKeyframe>
+                          </BehaviorLayer>
+                        </Timeline>
+                        <Resource name="Audio player" type="Lock" timeout="0" />
+                      </Box>
+                      <Link inputowner="1" indexofinput="2" outputowner="0" indexofoutput="1" />
+                    </Diagram>
+                  </BehaviorKeyframe>
+                </BehaviorLayer>
+                <ActuatorList model="nao">
+                  <ActuatorCurve name="value" actuator="HeadYaw" mute="0" unit="0">
+                    <Key frame="15" value="0" />
+                    <Key frame="25" value="4.82967e-06" />
+                    <Key frame="35" value="4.82968e-06" />
+                    <Key frame="45" value="4.82968e-06" />
+                    <Key frame="55" value="-2.73208e-05" />
+                    <Key frame="65" value="18" />
+                    <Key frame="75" value="-17" />
+                    <Key frame="85" value="-68" />
+                    <Key frame="95" value="-16" />
+                    <Key frame="105" value="12" />
+                    <Key frame="118" value="90" />
+                    <Key frame="131" value="12" />
+                    <Key frame="142" value="8" />
+                    <Key frame="152" value="0" />
+                    <Key frame="162" value="-8" />
+                    <Key frame="172" value="16" />
+                    <Key frame="185" value="-12" />
+                    <Key frame="198" value="-90" />
+                    <Key frame="211" value="-12" />
+                    <Key frame="222" value="-8" />
+                    <Key frame="231" value="0" />
+                    <Key frame="250" value="0" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="HeadPitch" mute="0" unit="0">
+                    <Key frame="15" value="0" />
+                    <Key frame="25" value="5.12931e-06" />
+                    <Key frame="35" value="-2.73208e-05" />
+                    <Key frame="45" value="5.0962e-06" />
+                    <Key frame="55" value="6.01466e-06" />
+                    <Key frame="65" value="19" />
+                    <Key frame="75" value="18" />
+                    <Key frame="85" value="5.26559e-06" />
+                    <Key frame="95" value="-19" />
+                    <Key frame="105" value="8" />
+                    <Key frame="118" value="-5" />
+                    <Key frame="131" value="8" />
+                    <Key frame="142" value="22" />
+                    <Key frame="152" value="32" />
+                    <Key frame="162" value="22" />
+                    <Key frame="172" value="-19" />
+                    <Key frame="185" value="8" />
+                    <Key frame="198" value="-5" />
+                    <Key frame="211" value="8" />
+                    <Key frame="222" value="22" />
+                    <Key frame="231" value="0" />
+                    <Key frame="250" value="-10.0411" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LShoulderPitch" mute="0" unit="0">
+                    <Key frame="15" value="90" />
+                    <Key frame="25" value="110" />
+                    <Key frame="35" value="120" />
+                    <Key frame="45" value="90" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="21" />
+                    <Key frame="75" value="20" />
+                    <Key frame="85" value="11" />
+                    <Key frame="95" value="-46" />
+                    <Key frame="105" value="-10" />
+                    <Key frame="118" value="-17" />
+                    <Key frame="131" value="-10" />
+                    <Key frame="142" value="30" />
+                    <Key frame="152" value="27" />
+                    <Key frame="162" value="19" />
+                    <Key frame="172" value="-27" />
+                    <Key frame="185" value="4" />
+                    <Key frame="198" value="-4" />
+                    <Key frame="211" value="4" />
+                    <Key frame="222" value="19" />
+                    <Key frame="231" value="97" />
+                    <Key frame="250" value="83.0164" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LShoulderRoll" mute="0" unit="0">
+                    <Key frame="15" value="10" />
+                    <Key frame="25" value="20" />
+                    <Key frame="35" value="10" />
+                    <Key frame="45" value="10" />
+                    <Key frame="55" value="10" />
+                    <Key frame="65" value="40" />
+                    <Key frame="75" value="0" />
+                    <Key frame="85" value="5" />
+                    <Key frame="95" value="10" />
+                    <Key frame="105" value="23" />
+                    <Key frame="118" value="66" />
+                    <Key frame="131" value="23" />
+                    <Key frame="142" value="23" />
+                    <Key frame="152" value="10" />
+                    <Key frame="162" value="0" />
+                    <Key frame="172" value="23" />
+                    <Key frame="185" value="0" />
+                    <Key frame="198" value="0" />
+                    <Key frame="211" value="0" />
+                    <Key frame="222" value="12" />
+                    <Key frame="231" value="54" />
+                    <Key frame="250" value="12.7723" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LElbowYaw" mute="0" unit="0">
+                    <Key frame="15" value="-90" />
+                    <Key frame="25" value="-90" />
+                    <Key frame="35" value="-90" />
+                    <Key frame="45" value="-90" />
+                    <Key frame="55" value="-90" />
+                    <Key frame="65" value="-22" />
+                    <Key frame="75" value="0" />
+                    <Key frame="85" value="0" />
+                    <Key frame="95" value="0" />
+                    <Key frame="105" value="0" />
+                    <Key frame="118" value="0" />
+                    <Key frame="131" value="0" />
+                    <Key frame="142" value="0" />
+                    <Key frame="152" value="12" />
+                    <Key frame="162" value="11" />
+                    <Key frame="172" value="-24" />
+                    <Key frame="185" value="-24" />
+                    <Key frame="198" value="-5" />
+                    <Key frame="211" value="-24" />
+                    <Key frame="222" value="11" />
+                    <Key frame="227" value="-21.7" />
+                    <Key frame="231" value="-14" />
+                    <Key frame="250" value="-68.8135" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LElbowRoll" mute="0" unit="0">
+                    <Key frame="15" value="0" />
+                    <Key frame="25" value="-40" />
+                    <Key frame="35" value="-60" />
+                    <Key frame="45" value="0" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="-95" />
+                    <Key frame="75" value="-55" />
+                    <Key frame="85" value="-85" />
+                    <Key frame="95" value="-58" />
+                    <Key frame="105" value="-58" />
+                    <Key frame="118" value="0" />
+                    <Key frame="131" value="-58" />
+                    <Key frame="142" value="-58" />
+                    <Key frame="152" value="-51" />
+                    <Key frame="162" value="-49" />
+                    <Key frame="172" value="-64" />
+                    <Key frame="185" value="-49" />
+                    <Key frame="198" value="-72" />
+                    <Key frame="211" value="-49" />
+                    <Key frame="222" value="-49" />
+                    <Key frame="227" value="-57" />
+                    <Key frame="231" value="-81.4" />
+                    <Key frame="250" value="-23.6757" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LWristYaw" mute="0" unit="0">
+                    <Key frame="15" value="-88" />
+                    <Key frame="250" value="5.78932" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LHand" mute="0" unit="1">
+                    <Key frame="15" value="0" />
+                    <Key frame="250" value="0.291787" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RShoulderPitch" mute="0" unit="0">
+                    <Key frame="15" value="90" />
+                    <Key frame="25" value="110" />
+                    <Key frame="35" value="120" />
+                    <Key frame="45" value="90" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="10" />
+                    <Key frame="75" value="35" />
+                    <Key frame="85" value="60" />
+                    <Key frame="95" value="-27" />
+                    <Key frame="105" value="4" />
+                    <Key frame="118" value="-4" />
+                    <Key frame="131" value="4" />
+                    <Key frame="142" value="19" />
+                    <Key frame="152" value="27" />
+                    <Key frame="162" value="30" />
+                    <Key frame="172" value="-46" />
+                    <Key frame="185" value="-10" />
+                    <Key frame="198" value="-17" />
+                    <Key frame="211" value="-10" />
+                    <Key frame="222" value="30" />
+                    <Key frame="231" value="97" />
+                    <Key frame="250" value="82.7438" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RShoulderRoll" mute="0" unit="0">
+                    <Key frame="15" value="-10" />
+                    <Key frame="25" value="-10" />
+                    <Key frame="35" value="-20" />
+                    <Key frame="45" value="-10" />
+                    <Key frame="55" value="-9.99896" />
+                    <Key frame="65" value="-4" />
+                    <Key frame="75" value="-48" />
+                    <Key frame="85" value="-87" />
+                    <Key frame="95" value="-23" />
+                    <Key frame="105" value="0" />
+                    <Key frame="118" value="0" />
+                    <Key frame="131" value="0" />
+                    <Key frame="142" value="0" />
+                    <Key frame="152" value="-10" />
+                    <Key frame="162" value="-23" />
+                    <Key frame="172" value="-10" />
+                    <Key frame="185" value="-23" />
+                    <Key frame="198" value="-66" />
+                    <Key frame="211" value="-23" />
+                    <Key frame="222" value="-32" />
+                    <Key frame="231" value="-54" />
+                    <Key frame="250" value="-12.6315" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RElbowYaw" mute="0" unit="0">
+                    <Key frame="15" value="90" />
+                    <Key frame="25" value="90" />
+                    <Key frame="35" value="90" />
+                    <Key frame="45" value="90" />
+                    <Key frame="55" value="90" />
+                    <Key frame="65" value="11" />
+                    <Key frame="75" value="20" />
+                    <Key frame="85" value="90" />
+                    <Key frame="95" value="24" />
+                    <Key frame="105" value="24" />
+                    <Key frame="118" value="5" />
+                    <Key frame="131" value="24" />
+                    <Key frame="142" value="-11" />
+                    <Key frame="152" value="-12" />
+                    <Key frame="162" value="0" />
+                    <Key frame="172" value="0" />
+                    <Key frame="185" value="0" />
+                    <Key frame="198" value="0" />
+                    <Key frame="211" value="0" />
+                    <Key frame="222" value="0" />
+                    <Key frame="227" value="19.6" />
+                    <Key frame="231" value="14" />
+                    <Key frame="250" value="68.8565" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RElbowRoll" mute="0" unit="0">
+                    <Key frame="15" value="0" />
+                    <Key frame="25" value="40" />
+                    <Key frame="35" value="60" />
+                    <Key frame="45" value="1.47493e-05" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="71" />
+                    <Key frame="75" value="94" />
+                    <Key frame="85" value="4" />
+                    <Key frame="95" value="64" />
+                    <Key frame="105" value="49" />
+                    <Key frame="118" value="72" />
+                    <Key frame="131" value="49" />
+                    <Key frame="142" value="49" />
+                    <Key frame="152" value="51" />
+                    <Key frame="162" value="58" />
+                    <Key frame="172" value="58" />
+                    <Key frame="185" value="58" />
+                    <Key frame="198" value="2" />
+                    <Key frame="211" value="58" />
+                    <Key frame="222" value="58" />
+                    <Key frame="227" value="64.9" />
+                    <Key frame="231" value="78.3" />
+                    <Key frame="250" value="23.387" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RWristYaw" mute="0" unit="0">
+                    <Key frame="15" value="88" />
+                    <Key frame="250" value="6.23278" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RHand" mute="0" unit="1">
+                    <Key frame="15" value="0" />
+                    <Key frame="250" value="0.301984" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LHipYawPitch" mute="0" unit="0">
+                    <Key frame="15" value="-6.2" />
+                    <Key frame="25" value="-6.9" />
+                    <Key frame="35" value="-6.94104" />
+                    <Key frame="45" value="-7.04938" />
+                    <Key frame="55" value="-8.2" />
+                    <Key frame="65" value="-9.6" />
+                    <Key frame="75" value="-5.7" />
+                    <Key frame="85" value="0" />
+                    <Key frame="95" value="0" />
+                    <Key frame="105" value="0" />
+                    <Key frame="118" value="0" />
+                    <Key frame="131" value="0" />
+                    <Key frame="142" value="-0.566539" />
+                    <Key frame="152" value="0" />
+                    <Key frame="162" value="0" />
+                    <Key frame="172" value="0" />
+                    <Key frame="185" value="0" />
+                    <Key frame="198" value="0" />
+                    <Key frame="211" value="0" />
+                    <Key frame="222" value="-3.9" />
+                    <Key frame="231" value="0" />
+                    <Key frame="250" value="-9.74084" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LHipRoll" mute="0" unit="0">
+                    <Key frame="15" value="-3" />
+                    <Key frame="25" value="-7" />
+                    <Key frame="35" value="-11.4235" />
+                    <Key frame="45" value="5.96288" />
+                    <Key frame="55" value="6" />
+                    <Key frame="65" value="-10" />
+                    <Key frame="75" value="10" />
+                    <Key frame="85" value="24.1" />
+                    <Key frame="95" value="30.3" />
+                    <Key frame="105" value="28.7" />
+                    <Key frame="118" value="32.4" />
+                    <Key frame="131" value="28.7" />
+                    <Key frame="142" value="19.9829" />
+                    <Key frame="152" value="0" />
+                    <Key frame="162" value="-15" />
+                    <Key frame="167" value="14.4" />
+                    <Key frame="172" value="15" />
+                    <Key frame="185" value="8" />
+                    <Key frame="198" value="40" />
+                    <Key frame="211" value="8" />
+                    <Key frame="222" value="-15" />
+                    <Key frame="231" value="0" />
+                    <Key frame="250" value="6.82441" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LHipPitch" mute="0" unit="0">
+                    <Key frame="15" value="0" />
+                    <Key frame="25" value="0" />
+                    <Key frame="35" value="-19.8395" />
+                    <Key frame="45" value="-0.0382666" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="-40" />
+                    <Key frame="75" value="-35" />
+                    <Key frame="85" value="-60" />
+                    <Key frame="95" value="-60" />
+                    <Key frame="105" value="-60" />
+                    <Key frame="118" value="-60" />
+                    <Key frame="131" value="-60" />
+                    <Key frame="142" value="-59.6787" />
+                    <Key frame="152" value="-50" />
+                    <Key frame="162" value="-42.5" />
+                    <Key frame="172" value="-7" />
+                    <Key frame="185" value="-50" />
+                    <Key frame="198" value="0" />
+                    <Key frame="211" value="-50" />
+                    <Key frame="222" value="-37.5" />
+                    <Key frame="231" value="-60" />
+                    <Key frame="250" value="7.30057" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LKneePitch" mute="0" unit="0">
+                    <Key frame="15" value="0" />
+                    <Key frame="25" value="0" />
+                    <Key frame="35" value="42.8009" />
+                    <Key frame="45" value="0.0951225" />
+                    <Key frame="55" value="-5.69182e-06" />
+                    <Key frame="65" value="80" />
+                    <Key frame="75" value="70" />
+                    <Key frame="85" value="120" />
+                    <Key frame="95" value="120" />
+                    <Key frame="105" value="120" />
+                    <Key frame="118" value="120" />
+                    <Key frame="131" value="120" />
+                    <Key frame="142" value="120.887" />
+                    <Key frame="152" value="100" />
+                    <Key frame="162" value="85" />
+                    <Key frame="172" value="7" />
+                    <Key frame="185" value="100" />
+                    <Key frame="198" value="0" />
+                    <Key frame="211" value="100" />
+                    <Key frame="222" value="75" />
+                    <Key frame="231" value="120" />
+                    <Key frame="250" value="-5.09006" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LAnklePitch" mute="0" unit="0">
+                    <Key frame="15" value="5.75265e-06" />
+                    <Key frame="25" value="0" />
+                    <Key frame="35" value="-22.1511" />
+                    <Key frame="45" value="-0.569877" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="-37.1" />
+                    <Key frame="75" value="-35" />
+                    <Key frame="85" value="-60" />
+                    <Key frame="95" value="-60" />
+                    <Key frame="105" value="-60" />
+                    <Key frame="118" value="-60" />
+                    <Key frame="131" value="-60" />
+                    <Key frame="142" value="-59.6787" />
+                    <Key frame="152" value="-50" />
+                    <Key frame="162" value="-42.5" />
+                    <Key frame="172" value="0" />
+                    <Key frame="185" value="5.75265e-06" />
+                    <Key frame="198" value="30" />
+                    <Key frame="211" value="5.75265e-06" />
+                    <Key frame="217" value="-31.8" />
+                    <Key frame="222" value="-37.5" />
+                    <Key frame="231" value="-60" />
+                    <Key frame="250" value="5.00876" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="LAnkleRoll" mute="0" unit="0">
+                    <Key frame="15" value="3" />
+                    <Key frame="25" value="7" />
+                    <Key frame="35" value="9.83048" />
+                    <Key frame="45" value="-6.94104" />
+                    <Key frame="55" value="-6" />
+                    <Key frame="65" value="10" />
+                    <Key frame="75" value="-15" />
+                    <Key frame="85" value="3.6" />
+                    <Key frame="95" value="4.2" />
+                    <Key frame="105" value="0.5" />
+                    <Key frame="118" value="1.8" />
+                    <Key frame="131" value="0.5" />
+                    <Key frame="142" value="6.97225" />
+                    <Key frame="152" value="0" />
+                    <Key frame="162" value="-13.8" />
+                    <Key frame="167" value="-31.9" />
+                    <Key frame="172" value="-24.3" />
+                    <Key frame="185" value="-20" />
+                    <Key frame="198" value="0" />
+                    <Key frame="211" value="-20" />
+                    <Key frame="222" value="-17.9" />
+                    <Key frame="231" value="0" />
+                    <Key frame="250" value="-6.34798" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RHipRoll" mute="0" unit="0">
+                    <Key frame="15" value="3" />
+                    <Key frame="25" value="-7" />
+                    <Key frame="35" value="-24.8982" />
+                    <Key frame="45" value="-6.04821" />
+                    <Key frame="55" value="-6" />
+                    <Key frame="65" value="-20" />
+                    <Key frame="75" value="-45" />
+                    <Key frame="85" value="-31" />
+                    <Key frame="95" value="-12.7" />
+                    <Key frame="105" value="-25.6" />
+                    <Key frame="118" value="-40" />
+                    <Key frame="131" value="-25.6" />
+                    <Key frame="142" value="15.2077" />
+                    <Key frame="152" value="0" />
+                    <Key frame="162" value="-20" />
+                    <Key frame="172" value="-30.9" />
+                    <Key frame="185" value="-29.3" />
+                    <Key frame="198" value="-34.2" />
+                    <Key frame="211" value="-35" />
+                    <Key frame="222" value="-30.5" />
+                    <Key frame="231" value="0" />
+                    <Key frame="250" value="-6.82407" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RHipPitch" mute="0" unit="0">
+                    <Key frame="15" value="0" />
+                    <Key frame="25" value="0" />
+                    <Key frame="35" value="0.134711" />
+                    <Key frame="45" value="0.21464" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="-40" />
+                    <Key frame="75" value="-10" />
+                    <Key frame="85" value="-6" />
+                    <Key frame="95" value="-7" />
+                    <Key frame="105" value="-50" />
+                    <Key frame="118" value="0" />
+                    <Key frame="131" value="-50" />
+                    <Key frame="142" value="-45.3545" />
+                    <Key frame="152" value="-50" />
+                    <Key frame="162" value="-60" />
+                    <Key frame="172" value="-60" />
+                    <Key frame="185" value="-60" />
+                    <Key frame="198" value="-60" />
+                    <Key frame="211" value="-60" />
+                    <Key frame="222" value="-60" />
+                    <Key frame="231" value="-60" />
+                    <Key frame="250" value="7.30058" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RKneePitch" mute="0" unit="0">
+                    <Key frame="15" value="0" />
+                    <Key frame="25" value="0" />
+                    <Key frame="35" value="0.178188" />
+                    <Key frame="45" value="-0.410883" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="80" />
+                    <Key frame="75" value="20" />
+                    <Key frame="85" value="7" />
+                    <Key frame="95" value="7" />
+                    <Key frame="105" value="100" />
+                    <Key frame="118" value="0" />
+                    <Key frame="131" value="100" />
+                    <Key frame="142" value="90.0035" />
+                    <Key frame="152" value="100" />
+                    <Key frame="162" value="120" />
+                    <Key frame="172" value="120" />
+                    <Key frame="185" value="120" />
+                    <Key frame="198" value="120" />
+                    <Key frame="211" value="120" />
+                    <Key frame="222" value="120" />
+                    <Key frame="231" value="120" />
+                    <Key frame="250" value="-5.26102" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RAnklePitch" mute="0" unit="0">
+                    <Key frame="15" value="5.75266e-06" />
+                    <Key frame="25" value="0" />
+                    <Key frame="35" value="0.266079" />
+                    <Key frame="45" value="-0.349163" />
+                    <Key frame="55" value="0" />
+                    <Key frame="65" value="-40" />
+                    <Key frame="75" value="-10" />
+                    <Key frame="85" value="0" />
+                    <Key frame="95" value="0" />
+                    <Key frame="105" value="5.75266e-06" />
+                    <Key frame="118" value="30" />
+                    <Key frame="131" value="5.75266e-06" />
+                    <Key frame="142" value="-45.8771" />
+                    <Key frame="152" value="-50" />
+                    <Key frame="162" value="-60" />
+                    <Key frame="172" value="-60" />
+                    <Key frame="185" value="-60" />
+                    <Key frame="198" value="-60" />
+                    <Key frame="211" value="-60" />
+                    <Key frame="222" value="-60" />
+                    <Key frame="231" value="-60" />
+                    <Key frame="250" value="5.00876" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="value" actuator="RAnkleRoll" mute="0" unit="0">
+                    <Key frame="15" value="-3" />
+                    <Key frame="25" value="7.5" />
+                    <Key frame="35" value="22.6325" />
+                    <Key frame="45" value="6.68218" />
+                    <Key frame="55" value="6" />
+                    <Key frame="65" value="16.9" />
+                    <Key frame="75" value="35.6" />
+                    <Key frame="85" value="45" />
+                    <Key frame="95" value="42.6" />
+                    <Key frame="105" value="25" />
+                    <Key frame="118" value="0" />
+                    <Key frame="131" value="25" />
+                    <Key frame="142" value="9.83317" />
+                    <Key frame="152" value="0" />
+                    <Key frame="162" value="-10" />
+                    <Key frame="172" value="-2.41531" />
+                    <Key frame="185" value="-3.18747" />
+                    <Key frame="198" value="-0.718272" />
+                    <Key frame="211" value="-0.558977" />
+                    <Key frame="222" value="-1.55043" />
+                    <Key frame="231" value="0" />
+                    <Key frame="250" value="6.34774" />
+                  </ActuatorCurve>
+                  <ActuatorCurve name="RHipYawPitch (RHipYawPitch)" actuator="RHipYawPitch" mute="0" unit="0">
+                    <Key frame="15" value="-6.2" />
+                    <Key frame="25" value="-6.9" />
+                    <Key frame="35" value="-7.55628" />
+                    <Key frame="45" value="-7.04938" />
+                    <Key frame="55" value="-8.2" />
+                    <Key frame="65" value="-9.6" />
+                    <Key frame="142" value="0.178188" />
+                    <Key frame="250" value="-9.74084" />
+                  </ActuatorCurve>
+                </ActuatorList>
+              </Timeline>
+              <Resource name="Standing" type="Lock" timeout="1" />
+              <Resource name="All motors" type="Lock" timeout="1" />
+            </Box>
+            <Link inputowner="11" indexofinput="2" outputowner="0" indexofoutput="2" />
+            <Link inputowner="0" indexofinput="4" outputowner="11" indexofoutput="4" />
+          </Diagram>
+        </BehaviorKeyframe>
+      </BehaviorLayer>
+    </Timeline>
+  </Box>
+</ChoregrapheProject>

gestures/5_tai_chi/translations/translation_en_US.ts

@@ -0,0 +1,3 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE TS>
+<TS version="2.1" language="en_US"/>

nao_video_server.py

@@ -0,0 +1,433 @@
+# -*- coding: utf-8 -*-
+"""
+nao_video_server.py - RUNS ON THE ROBOT, under its own Python 2.7.
+
+Opens a qi session to NAOqi over localhost (zero network hop - it's
+the same box) and serves TWO persistent, length-framed TCP streams -
+one for compressed video (JPEG), one for compressed audio (mu-law) -
+each on its own port/connection. They used to share one connection,
+but a single big video frame mid-send could block audio behind it for
+its whole transfer time, which is a bad tradeoff for something as
+latency-sensitive as audio. Splitting them means a slow video frame
+can never delay audio.
+
+The audio connection is bidirectional: this side keeps writing the
+robot's own mic (TYPE_AUDIO) to the client, while a reader thread on
+the same socket listens for TYPE_TALK chunks pushed the other way
+(push-to-talk audio from the client's mic) and plays them on the
+robot's speaker via TalkPlayer. Video stays one-way, server->client.
+
+Why not HTTP: the phone tunnel only forwards specific TCP ports, so
+this still rides over TCP - but a 5-byte binary header has none of
+HTTP's chunked/multipart-boundary parsing overhead, and it's trivial
+to keep alive and reconnect on a flaky link.
+
+Why mic capture lives here now: it used to be shipped raw (16-bit PCM,
+uncompressed) straight through NAOqi's own pub/sub over the qi RPC
+session the client holds - i.e. across the tunnel, uncompressed, the
+whole time. Subscribing to ALAudioDevice locally (same zero-hop trick
+as the camera) means only mu-law-compressed bytes ever cross the
+tunnel, half the size of raw PCM.
+
+Wire format, identical on both connections (all integers big-endian,
+via `struct`):
+    1 byte   type     0x00 HELLO   0x01 VIDEO   0x02 AUDIO   0x03 TALK
+    4 bytes  length
+    N bytes  payload
+        HELLO payload: JSON, informational only
+        VIDEO payload: JPEG bytes (video connection, server->client only)
+        AUDIO payload: mu-law 8-bit samples, mono (audio connection, server->client - robot's mic)
+        TALK  payload: mu-law 8-bit samples, mono (audio connection, client->server - push-to-talk)
+
+Under load (slow/lossy tunnel), video quality and frame rate step
+down automatically (see QUALITY_TIERS) and recover once sends are
+fast again - and since encoding is throttled to match, this also cuts
+CPU use on a congested link instead of just wasting it on frames that
+would've been discarded anyway.
+
+Needs the SDK's own env vars, since the bindings aren't on a login
+shell's default path:
+
+    export PYTHONPATH=/opt/aldebaran/lib/python2.7/site-packages
+    export LD_LIBRARY_PATH=/opt/aldebaran/lib
+    python2 nao_video_server.py
+
+Stdlib only (BaseHTTPServer/Flask are gone entirely now) plus qi,
+cv2, and numpy, which are already confirmed present on the robot.
+"""
+import collections
+import json
+import socket
+import struct
+import subprocess
+import threading
+import time
+
+import cv2
+import numpy as np
+import qi
+
+# ---- tunables ----
+NAOQI_PORT = 9561          # local qi session port (confirmed working)
+VIDEO_PORT = 8000          # forward both of these through the phone tunnel
+AUDIO_PORT = 8001          # kept on its own connection so a slow video
+                            # frame can never block/delay audio delivery
+CAMERA_RESOLUTION = 1      # 0=kQQVGA(160x120) 1=kQVGA(320x240) 2=kVGA(640x480)
+CAMERA_COLORSPACE = 11     # kRGBColorSpace
+CAMERA_FPS = 20
+AUDIO_RATE = 16000
+AUDIO_CHANNELS = 1
+MIC_CHANNEL = 1            # 1=left 2=right 3=front 4=rear (ALAudioDevice channel index)
+AUDIO_QUEUE_MAX = 60       # chunks buffered before we start dropping the oldest
+
+# (jpeg_quality, min_seconds_between_frames) - best link first, worst last.
+# The server steps down a tier when a video send is slow, and back up
+# after a run of fast ones.
+QUALITY_TIERS = [
+    (70, 0.05),
+    (55, 0.07),
+    (40, 0.10),
+    (25, 0.15),
+]
+SLOW_SEND_THRESHOLD = 0.20        # seconds - a send this slow means congestion
+FAST_STREAK_TO_RECOVER = 30       # consecutive quick sends before stepping back up
+
+TYPE_HELLO = 0x00
+TYPE_VIDEO = 0x01
+TYPE_AUDIO = 0x02
+TYPE_TALK = 0x03
+
+
+def encode_ulaw(pcm16):
+    """int16 numpy array -> uint8 numpy array, mu-law companded.
+    Must match decode_ulaw() in naowalk.py exactly, or the far end
+    just hears noise - keep the two in sync if you touch this."""
+    mu = 255.0
+    x = np.clip(pcm16.astype(np.float64) / 32768.0, -1.0, 1.0)
+    y = np.sign(x) * np.log1p(mu * np.abs(x)) / np.log1p(mu)
+    return (((y + 1.0) / 2.0) * 255.0).round().astype(np.uint8)
+
+
+def decode_ulaw(q):
+    """uint8 mu-law samples -> int16 PCM. Inverse of encode_ulaw()
+    above, used to play back incoming push-to-talk (TYPE_TALK) chunks.
+    Must match naowalk.py's encode_ulaw() exactly, or push-to-talk
+    just produces noise on the robot's speaker."""
+    mu = 255.0
+    y = (q.astype(np.float64) / 255.0) * 2.0 - 1.0
+    x = np.sign(y) * (1.0 / mu) * (np.power(1.0 + mu, np.abs(y)) - 1.0)
+    return np.clip(x * 32768.0, -32768, 32767).astype(np.int16)
+
+
+class FrameGrabber(object):
+    """Background thread: keeps the single most recent JPEG-encoded
+    frame available, at whatever quality tier the link currently
+    supports. A stalled capture never blocks a client's read."""
+
+    def __init__(self, session):
+        self.video = session.service("ALVideoDevice")
+        self.client = self.video.subscribeCamera(
+            "MediaServer", 0, CAMERA_RESOLUTION, CAMERA_COLORSPACE, CAMERA_FPS)
+        self._lock = threading.Lock()
+        self._jpeg = None
+        self._tier = 0
+        self._fast_streak = 0
+        self._stop = threading.Event()
+        self._thread = threading.Thread(target=self._loop)
+        self._thread.daemon = True
+        self._thread.start()
+
+    def _loop(self):
+        last_encode = 0.0
+        while not self._stop.is_set():
+            try:
+                image = self.video.getImageRemote(self.client)
+                if not image or len(image) < 7:
+                    continue
+                # getImageRemote() blocks until the next camera frame is
+                # ready (~1/CAMERA_FPS), so this loop is already paced -
+                # but under a congested tier we send far less often than
+                # that, so skip the (expensive) convert+encode work for
+                # frames we're just going to overwrite unsent anyway.
+                quality, min_interval = self.quality_and_interval()
+                now = time.time()
+                if now - last_encode < min_interval:
+                    continue
+                w, h = image[0], image[1]
+                arr = np.frombuffer(bytearray(image[6]), dtype=np.uint8).reshape((h, w, 3))
+                bgr = cv2.cvtColor(arr, cv2.COLOR_RGB2BGR)
+                ok, buf = cv2.imencode(".jpg", bgr, [cv2.IMWRITE_JPEG_QUALITY, quality])
+                if ok:
+                    with self._lock:
+                        self._jpeg = buf.tobytes()
+                    last_encode = now
+            except Exception as e:
+                print("frame grab error: %s" % e)
+                time.sleep(0.2)
+
+    def quality_and_interval(self):
+        with self._lock:
+            return QUALITY_TIERS[self._tier]
+
+    def report_send_time(self, elapsed):
+        """Called by the writer thread after each video send so we can
+        adapt to how the link is actually behaving right now."""
+        with self._lock:
+            if elapsed > SLOW_SEND_THRESHOLD:
+                self._fast_streak = 0
+                if self._tier < len(QUALITY_TIERS) - 1:
+                    self._tier += 1
+                    print("link looks congested - dropping to quality tier %d" % self._tier)
+            else:
+                self._fast_streak += 1
+                if self._fast_streak >= FAST_STREAK_TO_RECOVER and self._tier > 0:
+                    self._tier -= 1
+                    self._fast_streak = 0
+                    print("link recovered - raising to quality tier %d" % self._tier)
+
+    def latest(self):
+        with self._lock:
+            return self._jpeg
+
+    def close(self):
+        self._stop.set()
+        try:
+            self.video.unsubscribe(self.client)
+        except Exception:
+            pass
+
+
+class MicGrabber(object):
+    """Subscribes to the robot's own mic locally (zero-hop, same trick
+    as FrameGrabber) and mu-law-encodes each chunk as it arrives. This
+    replaces the old design where raw PCM was pushed to the client
+    directly through NAOqi's pub/sub over the (tunneled) qi session."""
+
+    def __init__(self, session, channel=MIC_CHANNEL):
+        self.audio = session.service("ALAudioDevice")
+        self._svc_name = "MicRelay"
+        self._lock = threading.Lock()
+        self._chunks = collections.deque(maxlen=AUDIO_QUEUE_MAX)
+        session.registerService(self._svc_name, self)
+        time.sleep(0.5)  # let NAOqi's service directory propagate the
+                          # registration before ALAudioDevice looks it up
+                          # by name, or subscribe() fails to find it
+        self.audio.setClientPreferences(self._svc_name, AUDIO_RATE, channel, 0)
+        self.audio.subscribe(self._svc_name)
+
+    def processRemote(self, nbOfChannels, nbOfSamplesByChannel, timeStamp, buffer):
+        try:
+            pcm = np.frombuffer(bytes(buffer), dtype=np.int16)
+            chunk = encode_ulaw(pcm).tobytes()
+            with self._lock:
+                self._chunks.append(chunk)
+        except Exception as e:
+            print("mic encode error: %s" % e)
+
+    def pop_all(self):
+        """Drain everything queued right now, oldest first."""
+        with self._lock:
+            out = list(self._chunks)
+            self._chunks.clear()
+        return out
+
+    def close(self):
+        try:
+            self.audio.unsubscribe(self._svc_name)
+        except Exception:
+            pass
+
+
+class TalkPlayer(object):
+    """Decodes incoming TYPE_TALK chunks (push-to-talk audio pushed
+    from the client) and plays them on the robot's speaker.
+
+    Uses a persistent `aplay` subprocess fed raw PCM over its stdin -
+    the same playback mechanism the old SSH `nc -l | aplay` relay
+    used, just sourced from decoded network chunks instead of a raw
+    pipe. The process is kept open across calls (silence between
+    presses is fine); it's only recreated if it exited or hasn't
+    been started yet."""
+
+    def __init__(self, rate=AUDIO_RATE):
+        self.rate = rate
+        self._proc = None
+        self._lock = threading.Lock()
+
+    def _ensure_proc(self):
+        if self._proc is None or self._proc.poll() is not None:
+            self._proc = subprocess.Popen(
+                ["aplay", "-q", "-t", "raw", "-f", "S16_LE",
+                 "-c", str(AUDIO_CHANNELS), "-r", str(self.rate)],
+                stdin=subprocess.PIPE)
+
+    def play(self, ulaw_payload):
+        pcm = decode_ulaw(np.frombuffer(ulaw_payload, dtype=np.uint8))
+        with self._lock:
+            try:
+                self._ensure_proc()
+                self._proc.stdin.write(pcm.tobytes())
+                self._proc.stdin.flush()
+            except Exception as e:
+                print("talk playback error: %s" % e)
+                self._proc = None
+
+    def close(self):
+        with self._lock:
+            if self._proc is not None:
+                try:
+                    self._proc.stdin.close()
+                    self._proc.terminate()
+                except Exception:
+                    pass
+                self._proc = None
+
+
+def _recv_exact(sock, n):
+    """Read exactly n bytes from a TCP socket (recv() can return short
+    reads/chunks at any point - never assume one call gives you n).
+    Mirror of naowalk.py's _recv_exact()."""
+    buf = b""
+    while len(buf) < n:
+        chunk = sock.recv(n - len(buf))
+        if not chunk:
+            raise IOError("media relay connection closed")
+        buf += chunk
+    return buf
+
+
+def _send(conn, msg_type, payload):
+    conn.sendall(struct.pack(">BI", msg_type, len(payload)))
+    conn.sendall(payload)
+
+
+def serve_video(grabber, host, port):
+    """Video gets its own connection/thread - a slow sendall() here
+    (a big JPEG frame over a bad link) must never be able to delay
+    audio, which used to share this same socket and paid for it in
+    latency."""
+    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+    sock.bind((host, port))
+    sock.listen(1)
+    print("video server listening on :%d" % port)
+    while True:
+        conn, addr = sock.accept()
+        conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+        conn.settimeout(5.0)
+        print("video client connected: %s" % (addr,))
+        last_jpeg = None
+        last_send = 0.0
+        try:
+            _send(conn, TYPE_HELLO, json.dumps({"stream": "video"}).encode("utf-8"))
+            while True:
+                jpeg = grabber.latest()
+                _, min_interval = grabber.quality_and_interval()
+                now = time.time()
+                if jpeg is not None and jpeg is not last_jpeg and now - last_send >= min_interval:
+                    t0 = time.time()
+                    _send(conn, TYPE_VIDEO, jpeg)
+                    grabber.report_send_time(time.time() - t0)
+                    last_jpeg = jpeg
+                    last_send = now
+                else:
+                    time.sleep(0.01)
+        except Exception as e:
+            print("video client disconnected (%s)" % e)
+        finally:
+            try:
+                conn.close()
+            except Exception:
+                pass
+
+
+def _talk_reader(conn, player):
+    """Runs alongside the audio writer loop on the same connection -
+    the audio socket is bidirectional now, so this thread's the read
+    half: it pulls TYPE_TALK frames pushed from the client (push-to-
+    talk) and hands each decoded chunk to TalkPlayer. conn has a
+    socket timeout set by the caller, so a quiet stretch between
+    button presses just means occasional socket.timeout - not a
+    disconnect - and the loop keeps going."""
+    try:
+        while True:
+            try:
+                header = _recv_exact(conn, 5)
+            except socket.timeout:
+                continue
+            msg_type, length = struct.unpack(">BI", header)
+            payload = _recv_exact(conn, length)
+            if msg_type == TYPE_TALK:
+                player.play(payload)
+    except Exception as e:
+        print("talk reader stopped (%s)" % e)
+
+
+def serve_audio(mic, player, host, port):
+    """Audio's own connection/thread. Checked eagerly (short sleep,
+    not paced like video) since gaps are far more noticeable than a
+    video frame being a beat stale."""
+    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+    sock.bind((host, port))
+    sock.listen(1)
+    print("audio server listening on :%d" % port)
+    while True:
+        conn, addr = sock.accept()
+        conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+        conn.settimeout(5.0)
+        print("audio client connected: %s" % (addr,))
+        reader_thread = threading.Thread(target=_talk_reader, args=(conn, player))
+        reader_thread.daemon = True
+        reader_thread.start()
+        try:
+            _send(conn, TYPE_HELLO,
+                  json.dumps({"rate": AUDIO_RATE, "channels": AUDIO_CHANNELS}).encode("utf-8"))
+            while True:
+                chunks = mic.pop_all() if mic is not None else []
+                if chunks:
+                    for chunk in chunks:
+                        _send(conn, TYPE_AUDIO, chunk)
+                else:
+                    time.sleep(0.005)
+        except Exception as e:
+            print("audio client disconnected (%s)" % e)
+        finally:
+            try:
+                conn.close()
+            except Exception:
+                pass
+
+
+def main():
+    session = qi.Session()
+    session.connect("tcp://127.0.0.1:%d" % NAOQI_PORT)
+    print("connected to local NAOqi on port %d" % NAOQI_PORT)
+
+    grabber = FrameGrabber(session)
+    mic = None
+    try:
+        mic = MicGrabber(session)
+        print("mic capture ready (mu-law, %dHz)" % AUDIO_RATE)
+    except Exception as e:
+        print("mic capture unavailable (%s) - video only" % e)
+
+    video_thread = threading.Thread(target=serve_video, args=(grabber, "0.0.0.0", VIDEO_PORT))
+    video_thread.daemon = True
+    video_thread.start()
+
+    player = TalkPlayer()
+
+    try:
+        serve_audio(mic, player, "0.0.0.0", AUDIO_PORT)
+    except KeyboardInterrupt:
+        pass
+    finally:
+        grabber.close()
+        if mic:
+            mic.close()
+        player.close()
+
+
+if __name__ == "__main__":
+    main()

naomusic.py

@@ -0,0 +1,150 @@
+# naomusic.py
+# -*- encoding: UTF-8 -*-
+
+import subprocess
+import os
+import json
+import re
+import time
+import threading
+
+class NaoMusicPlayer:
+    def __init__(self, session, nao_ip, ssh_port=22):
+        self.session = session
+        self.player = session.service("ALAudioPlayer")
+        self.nao_ip = nao_ip
+        self.ssh_port = ssh_port
+        self.music_dir = "/home/nao/music"
+        self.local_path = "/tmp/song.mp3"
+        self.remote_path = f"{self.music_dir}/song.mp3"
+        try:
+            subprocess.run(["sshpass", "-p", "nao", "ssh", "-p", str(self.ssh_port),
+                             f"nao@{nao_ip}", f"mkdir -p {self.music_dir}"],
+                         stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+        except: pass
+        self.current_file = None
+        self.current_task_id = None
+        self.is_playing = False
+        self.current_title = ""
+        self.is_loading = False
+        self.load_error = None
+        self.status = ""
+        self.progress = 0.0
+
+    def search(self, query):
+        try:
+            cmd = ["yt-dlp", f"ytsearch5:{query}", "--flat-playlist", "--dump-json",
+                   "--no-warnings", "--quiet"]
+            result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
+            entries = []
+            for line in result.stdout.strip().splitlines():
+                if not line.strip():
+                    continue
+                try:
+                    data = json.loads(line)
+                except json.JSONDecodeError:
+                    continue
+                vid = data.get("id")
+                url = data.get("url")
+                if not url or not url.startswith("http"):
+                    url = f"https://www.youtube.com/watch?v={vid}" if vid else None
+                if url:
+                    entries.append({"url": url, "title": data.get("title", "Unknown")})
+            return entries
+        except Exception as e:
+            print(f"Search error: {e}")
+            return []
+
+    _DL_PROGRESS_RE = re.compile(r'\[download\]\s+([\d.]+)%')
+
+    def play(self, url, title):
+        self.stop()
+        self.current_title = title
+        self.is_loading = True
+        self.load_error = None
+        self.status = "Starting download..."
+        self.progress = 0.0
+        try:
+            if os.path.exists(self.local_path):
+                os.remove(self.local_path)
+        except: pass
+
+        try:
+            cmd = ["yt-dlp", url,
+                   "-x", "--audio-format", "mp3",
+                   "-o", self.local_path,
+                   "--no-playlist", "--no-warnings", "--newline"]
+            print(f"\n▶ yt-dlp {url}")
+            proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+                                     text=True, bufsize=1)
+            for line in proc.stdout:
+                line = line.rstrip()
+                if not line:
+                    continue
+                print(f"  [yt-dlp] {line}")
+                m = self._DL_PROGRESS_RE.search(line)
+                if m:
+                    self.progress = float(m.group(1))
+                    self.status = f"Downloading... {self.progress:.0f}%"
+                elif "ExtractAudio" in line or "Extracting audio" in line:
+                    self.status = "Converting to mp3..."
+                elif line.startswith("[download] Destination"):
+                    self.status = "Downloading..."
+            proc.wait(timeout=120)
+
+            if proc.returncode != 0 or not os.path.exists(self.local_path):
+                print(f"Download failed (exit {proc.returncode})")
+                self.load_error = "Download failed"
+                return False
+
+            size_mb = os.path.getsize(self.local_path) / (1024 * 1024)
+            print(f"▶ scp {self.local_path} -> nao@{self.nao_ip}:{self.remote_path} ({size_mb:.1f} MB)")
+            self.progress = 0.0
+            upload_start = time.time()
+            stop_ticker = threading.Event()
+
+            def ticker():
+                while not stop_ticker.is_set():
+                    elapsed = time.time() - upload_start
+                    self.status = f"Uploading to NAO ({size_mb:.1f} MB)... {elapsed:.0f}s"
+                    stop_ticker.wait(1.0)
+
+            t = threading.Thread(target=ticker, daemon=True)
+            t.start()
+            scp_result = subprocess.run(
+                ["sshpass", "-p", "nao", "scp", "-P", str(self.ssh_port),
+                 self.local_path, f"nao@{self.nao_ip}:{self.remote_path}"],
+                capture_output=True, text=True, timeout=60
+            )
+            stop_ticker.set()
+            t.join(timeout=1.0)
+            print(f"  [scp] exit {scp_result.returncode}" +
+                  (f": {scp_result.stderr.strip()}" if scp_result.stderr.strip() else ""))
+            if scp_result.returncode != 0:
+                print(f"SCP failed: {scp_result.stderr.strip()}")
+                self.load_error = "SCP failed"
+                return False
+
+            self.current_file = self.remote_path
+            self.status = "Playing"
+            self.progress = 100.0
+            self.current_task_id = self.player.playFile(self.remote_path)
+            self.is_playing = True
+            print(f"♪ Playing on NAO: {title} (task {self.current_task_id})")
+            return True
+        except Exception as e:
+            print(f"Playback error: {e}")
+            self.load_error = str(e)
+            return False
+        finally:
+            self.is_loading = False
+
+    def stop(self):
+        try:
+            self.player.stopAll()
+        except Exception as e:
+            print(f"Stop error: {e}")
+        self.is_playing = False
+        self.current_file = None
+        self.current_task_id = None
+        self.current_title = ""

naowifi.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+nmcli c up naorobotwifi\ \(WAP\) && nmcli c up BC-VR\ \(USBWIFI\)

readme.md

@@ -0,0 +1,24 @@
+# NAO Walk - Teleoperation + Music Player
+
+A full-featured Python teleoperation tool for the NAO robot with live camera feed, head control, microphone streaming to your laptop, and **music playback directly on the robot**.
+
+---
+
+## Features
+
+- **Live Camera Feed** from NAO (upscaled on PC)
+- **Keyboard Teleop** (WASD / Arrow keys)
+- **Head Control** (I J K L)
+- **Wave Gesture** (Key `1`)
+- **Live Mic Streaming** from NAO to your laptop (with adjustable gain)
+- **Music Player on NAO** — Search & play YouTube music directly through the robot's speakers
+- **TTS Chat** — Type messages for the robot to speak (`T`)
+- **Battery & Volume Display**
+
+---
+
+## Requirements
+
+### On your laptop:
+```bash
+pip install qi opencv-python numpy pygame yt-dlp

requirements.txt

@@ -1,12 +1,14 @@
-# requirements.txt for NAO Robot Teleoperation
+# Install with: pip install -r requirements.txt
+#
+# NOTE: the NAOqi Python SDK ("qi") is NOT on PyPI. It ships separately from
+# Aldebaran/SoftBank Robotics as a platform-specific wheel (pynaoqi-python*).
+# Download it from the Softbank Robotics / Aldebaran developer portal, match
+# it to your Python version/OS, and install it with:
+#   pip install ./pynaoqi-*.whl
+# See README.md for details.
 
-# Core NAOqi SDK (required - install via Aldebaran/SoftBank installer)
+opencv-python>=4.8
-# qi  # Usually installed with the NAOqi Python SDK
+numpy>=1.24
-
+pygame>=2.5
-# Python packages
+pyaudio>=0.2.13
-opencv-python>=4.5.0
+yt-dlp>=2024.1.1
-pygame>=2.0.0
-numpy>=1.21.0
-
-# Optional but recommended
-pillow>=9.0.0        # If you want PIL fallback for images

start.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+pactl set-source-volume alsa_input.pci-0000_00_1b.0.analog-stereo 32%
+#python3.11 naowalk.py --ip spike.local --port 9561 --mic-gain 9 --video-scale 3.0 
+#python3.11 naowalk.py --ip 127.0.0.1 --port 9561 --mic-gain 9 --video-scale 3.0 --nao-ssh-port 2222 --media-relay 127.0.0.1:8000
+python3.11 naowalk.py --ip spike.local --port 9561 --mic-gain 9 --video-scale 3.0 --nao-ssh-port 22 --media-relay spike.local:8000

startSSHTunnel.sh

@@ -0,0 +1 @@
+ssh -N -L 9561:192.168.0.100:9561 -L 2222:192.168.0.100:22 -L 8000:192.168.0.100:8000 root@10.196.124.46