Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| bf3e197a20 | |||
| a5cdb4e4bd |
@@ -0,0 +1,105 @@
|
|||||||
|
# naowalk
|
||||||
|
|
||||||
|
A keyboard teleop client for the NAO robot: live video, walking, head control,
|
||||||
|
two-way audio (hear the robot's mics, make it speak via TTS), and a YouTube
|
||||||
|
music player that downloads a track with `yt-dlp` and plays it through the
|
||||||
|
robot's own speakers.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
**System packages** (Debian/Ubuntu names — adjust for your OS):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install portaudio19-dev ffmpeg sshpass openssh-client
|
||||||
|
```
|
||||||
|
|
||||||
|
- `portaudio19-dev` — needed to build/install `pyaudio`
|
||||||
|
- `ffmpeg` — needed by `yt-dlp` to extract audio to mp3
|
||||||
|
- `sshpass` + `openssh-client` — used to `scp` downloaded tracks onto the
|
||||||
|
robot and to run `mkdir` over `ssh` (the script assumes the default
|
||||||
|
`nao`/`nao` credentials)
|
||||||
|
|
||||||
|
**NAOqi Python SDK (`qi`)** — not on PyPI. Download the `pynaoqi-python`
|
||||||
|
wheel that matches your OS/Python version from the Aldebaran/SoftBank
|
||||||
|
Robotics developer site and install it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install ./pynaoqi-*.whl
|
||||||
|
```
|
||||||
|
|
||||||
|
**Python packages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 naowalk.py --ip <robot-ip> [options]
|
||||||
|
```
|
||||||
|
|
||||||
|
| Flag | Default | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| `--ip` | `127.0.0.1` | Robot's IP address |
|
||||||
|
| `--port` | `9559` | NAOqi port |
|
||||||
|
| `--audio-device-index` | auto | Output device index for hearing the robot's mic on your machine |
|
||||||
|
| `--mic-channel` | `left` | Which robot mic to stream (`left`, `right`, `front`, `rear`) — pick a working one, see Troubleshooting |
|
||||||
|
| `--mic-gain` | `8.0` | Amplification applied to the streamed mic audio |
|
||||||
|
| `--video-scale` | `2.0` | Display window scale (base feed is 320x240) |
|
||||||
|
|
||||||
|
## Controls
|
||||||
|
|
||||||
|
| Key | Action |
|
||||||
|
|---|---|
|
||||||
|
| `W`/`A`/`S`/`D` or arrows | Walk forward/turn/back |
|
||||||
|
| `I`/`K`/`J`/`L` | Head pitch/yaw |
|
||||||
|
| `Space` | Reset head to center |
|
||||||
|
| `1` | Wave gesture |
|
||||||
|
| `M` | Open music search (type query, `Enter` to search, `↑↓` to select, `Enter`/`1`-`5` to play) |
|
||||||
|
| `P` | Pause / resume music (resume restarts the track from the beginning — see Notes) |
|
||||||
|
| `X` | Stop music |
|
||||||
|
| `-` / `=` | Volume down/up |
|
||||||
|
| `T` | Type a message for the robot to speak (TTS) |
|
||||||
|
| `Esc` | Quit (or cancel current mode) |
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
- **`SoundReceiver`** — registered as a NAOqi service that receives the
|
||||||
|
robot's mic stream and plays it out your local speakers via `pyaudio`, with
|
||||||
|
gain and a prebuffer to smooth out network jitter.
|
||||||
|
- **`NaoMusicPlayer`** — searches YouTube with `yt-dlp`, downloads and
|
||||||
|
converts the top match to mp3, `scp`s it to the robot, and plays it with
|
||||||
|
`ALAudioPlayer`.
|
||||||
|
- **`NaoTeleop`** — the main loop: pygame window for video/input, keyboard
|
||||||
|
handling, head/walk control, and wiring everything above together.
|
||||||
|
|
||||||
|
## Notes / design decisions
|
||||||
|
|
||||||
|
- **No seek.** Seeking was removed entirely — it added the most complexity
|
||||||
|
for the least-used feature. `P` is a simple play/stop toggle; resuming
|
||||||
|
restarts the track from the beginning rather than tracking a playback
|
||||||
|
position.
|
||||||
|
- **`moveInit()` on startup.** The robot used to need its walk buttons
|
||||||
|
jittered for a few seconds before its first walk would work. `run()` now
|
||||||
|
calls `ALMotionProxy::moveInit()` once at startup, which settles the walk
|
||||||
|
engine into the right pose before any real move command — this is the
|
||||||
|
documented NAOqi fix for exactly this symptom.
|
||||||
|
- **`_safe()` helper.** Most one-off NAOqi calls (setting angles, volume,
|
||||||
|
stopping movement, unsubscribing services) are wrapped in a small
|
||||||
|
`self._safe(label, fn)` helper instead of a repeated `try/except` block, so
|
||||||
|
one failed call logs and moves on without crashing the teleop loop or the
|
||||||
|
shutdown sequence.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
- **Mic sounds dead / silent on one channel:** some robots have one or more
|
||||||
|
physical mics that produce no signal. Test all four with a quick energy
|
||||||
|
check, then pick a working channel with `--mic-channel`.
|
||||||
|
- **Robot falls over on the very first walk command:** make sure you're on a
|
||||||
|
build that includes the `moveInit()` startup call described above; if it's
|
||||||
|
still happening, increase the `time.sleep(1.0)` after `goToPosture` to give
|
||||||
|
the posture more time to settle first.
|
||||||
|
- **`scp`/`ssh` failures for music:** confirm the robot is reachable at
|
||||||
|
`--ip` and that `nao`/`nao` are still the correct credentials; the script
|
||||||
|
hardcodes them via `sshpass`.
|
||||||
+56
@@ -0,0 +1,56 @@
|
|||||||
|
---
|
||||||
|
title: "Replacing music player backend with yt-dlp"
|
||||||
|
tags: []
|
||||||
|
author: claude
|
||||||
|
count: 2
|
||||||
|
exporter: 3.1.0
|
||||||
|
date: 2026-07-16T17-20-34+0000
|
||||||
|
url: https://claude.ai/chat/8bb5e420-7ad2-4048-be3f-95894713aff6
|
||||||
|
---
|
||||||
|
|
||||||
|
# Replacing music player backend with yt-dlp
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
- [1: lets just gut the seek function and focus on what works. we also need…](#chat-1)
|
||||||
|
- [2: alright awesome, lets remove the start from current position also, i w…](#chat-2)
|
||||||
|
|
||||||
|
## chat-1
|
||||||
|
|
||||||
|
> lets just gut the seek function and focus on what works. we also need to address the robot falling over if i try to walk from standstill, but only the first time i do it. If i jitter the walk forward and back buttons for about 3 seconds to get over the initial walk issues, it can start walking without falling over. how do we fix this
|
||||||
|
|
||||||
|
Both very solvable. Let's do the walk fix first since it's the more important one — confirmed via NAOqi docs: `ALMotionProxy::moveInit()` is the standard fix for exactly this. It's a one-time blocking call that initializes the walk engine and settles the robot into the right pose _before_ the first real move command, which is exactly what your button-jittering was accidentally doing by brute force. It's missing from the script entirely.
|
||||||
|
|
||||||
|
Now let's gut `seek()` entirely per your request — but keep the position tracking it shares with `pause()`, since that's still needed for resume-from-position to work:
|
||||||
|
|
||||||
|
Right, orphaned body left over. Let's clean that up:
|
||||||
|
|
||||||
|
Now remove the comma/period key bindings that called `seek()`:
|
||||||
|
|
||||||
|
Now compile-check and confirm nothing else references `seek`:
|
||||||
|
|
||||||
|
Clean. Let's ship it:
|
||||||
|
|
||||||
|
___
|
||||||
|
###### [top](#table-of-contents)
|
||||||
|
## chat-2
|
||||||
|
|
||||||
|
> alright awesome, lets remove the start from current position also, i want to get the lines of code down, what can be done without compromising on the stability and functionality to simplify our code
|
||||||
|
|
||||||
|
Now let's do the full pass. First, the concrete simplification you asked for — remove position tracking, revert pause/resume to restart-from-beginning:
|
||||||
|
|
||||||
|
Now `pause()` becomes a plain stop/restart toggle — no position math, no `playFileFromPosition`:
|
||||||
|
|
||||||
|
Now let's cut the repetitive `try/except Exception as e: print(...)` boilerplate scattered through `NaoTeleop` — same safety, less code, via one small helper:
|
||||||
|
|
||||||
|
Now let's add the `_safe` helper and apply it across these repeated sites:
|
||||||
|
|
||||||
|
Now the `moveInit` startup call:
|
||||||
|
|
||||||
|
Now the shutdown block — this is where it pays off most:
|
||||||
|
|
||||||
|
Let's check the movement block and the `T` key's `stopMove()` call too:
|
||||||
|
|
||||||
|
___
|
||||||
|
###### [top](#table-of-contents)
|
||||||
|
|
||||||
+41
-64
@@ -147,7 +147,6 @@ class NaoMusicPlayer:
|
|||||||
self.current_file = None
|
self.current_file = None
|
||||||
self.current_task_id = None
|
self.current_task_id = None
|
||||||
self.is_playing = False
|
self.is_playing = False
|
||||||
self.paused_position = 0.0
|
|
||||||
self.current_title = ""
|
self.current_title = ""
|
||||||
self.is_loading = False
|
self.is_loading = False
|
||||||
self.load_error = None
|
self.load_error = None
|
||||||
@@ -262,25 +261,27 @@ class NaoMusicPlayer:
|
|||||||
self.is_loading = False
|
self.is_loading = False
|
||||||
|
|
||||||
def pause(self):
|
def pause(self):
|
||||||
|
"""Toggle playback. No position tracking: resuming restarts the track from the top."""
|
||||||
if self.current_task_id is None or not self.current_file:
|
if self.current_task_id is None or not self.current_file:
|
||||||
print("⏸ Nothing loaded to pause/resume")
|
print("⏸ Nothing loaded to pause/resume")
|
||||||
return
|
return
|
||||||
try:
|
if self.is_playing:
|
||||||
if self.is_playing:
|
try:
|
||||||
pos = self.player.getCurrentPosition(self.current_task_id)
|
|
||||||
self.player.stopAll()
|
self.player.stopAll()
|
||||||
self.paused_position = max(0.0, pos)
|
except Exception as e:
|
||||||
self.is_playing = False
|
print(f"stopAll error while pausing: {e}")
|
||||||
print(f"♪ Paused at {self.paused_position:.1f}s")
|
self.load_error = f"Pause error: {e}"
|
||||||
else:
|
return
|
||||||
self.current_task_id = self.player.playFileFromPosition(
|
self.is_playing = False
|
||||||
self.current_file, self.paused_position, 1.0, 0.0
|
print("♪ Paused")
|
||||||
)
|
else:
|
||||||
|
try:
|
||||||
|
self.current_task_id = self.player.playFile(self.current_file)
|
||||||
self.is_playing = True
|
self.is_playing = True
|
||||||
print(f"♪ Resumed from {self.paused_position:.1f}s (task {self.current_task_id})")
|
print(f"♪ Resumed from start (task {self.current_task_id})")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f"Pause/resume error: {e}")
|
print(f"Resume error: {e}")
|
||||||
self.load_error = f"Pause error: {e}"
|
self.load_error = f"Resume error: {e}"
|
||||||
|
|
||||||
def stop(self):
|
def stop(self):
|
||||||
try:
|
try:
|
||||||
@@ -290,16 +291,7 @@ class NaoMusicPlayer:
|
|||||||
self.is_playing = False
|
self.is_playing = False
|
||||||
self.current_file = None
|
self.current_file = None
|
||||||
self.current_task_id = None
|
self.current_task_id = None
|
||||||
self.paused_position = 0.0
|
self.current_title = ""
|
||||||
|
|
||||||
def seek(self, seconds):
|
|
||||||
if self.current_task_id is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
pos = self.player.getCurrentPosition(self.current_task_id)
|
|
||||||
self.player.goTo(self.current_task_id, max(0.0, pos + seconds))
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Seek error: {e}")
|
|
||||||
|
|
||||||
|
|
||||||
# ==========================================
|
# ==========================================
|
||||||
@@ -356,6 +348,14 @@ class NaoTeleop:
|
|||||||
self.clock = pygame.time.Clock()
|
self.clock = pygame.time.Clock()
|
||||||
self.update_title()
|
self.update_title()
|
||||||
|
|
||||||
|
def _safe(self, label, fn):
|
||||||
|
"""Run fn(), log+swallow any exception so one failure can't crash the loop."""
|
||||||
|
try:
|
||||||
|
return fn()
|
||||||
|
except Exception as e:
|
||||||
|
print(f"{label} error: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
def _init_audio_device(self):
|
def _init_audio_device(self):
|
||||||
try:
|
try:
|
||||||
self.audio_device = self.session.service("ALAudioDevice")
|
self.audio_device = self.session.service("ALAudioDevice")
|
||||||
@@ -426,18 +426,13 @@ class NaoTeleop:
|
|||||||
if keys[pygame.K_j]: self.head_yaw = min(self.head_yaw + speed, 2.0); changed = True
|
if keys[pygame.K_j]: self.head_yaw = min(self.head_yaw + speed, 2.0); changed = True
|
||||||
if keys[pygame.K_l]: self.head_yaw = max(self.head_yaw - speed, -2.0); changed = True
|
if keys[pygame.K_l]: self.head_yaw = max(self.head_yaw - speed, -2.0); changed = True
|
||||||
if changed:
|
if changed:
|
||||||
try:
|
self._safe("Head move", lambda: self.motion.setAngles(
|
||||||
self.motion.setAngles(["HeadYaw", "HeadPitch"], [self.head_yaw, self.head_pitch], 0.3)
|
["HeadYaw", "HeadPitch"], [self.head_yaw, self.head_pitch], 0.3))
|
||||||
except Exception as e:
|
|
||||||
print(f"Head move error: {e}")
|
|
||||||
|
|
||||||
def reset_head(self):
|
def reset_head(self):
|
||||||
self.head_yaw = 0.0
|
self.head_yaw = 0.0
|
||||||
self.head_pitch = 0.0
|
self.head_pitch = 0.0
|
||||||
try:
|
self._safe("Head reset", lambda: self.motion.setAngles(["HeadYaw", "HeadPitch"], [0.0, 0.0], 0.4))
|
||||||
self.motion.setAngles(["HeadYaw", "HeadPitch"], [0.0, 0.0], 0.4)
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Head reset error: {e}")
|
|
||||||
|
|
||||||
def update_title(self):
|
def update_title(self):
|
||||||
mode = "[TYPING] " if self.typing_mode else ""
|
mode = "[TYPING] " if self.typing_mode else ""
|
||||||
@@ -446,10 +441,7 @@ class NaoTeleop:
|
|||||||
def change_volume(self, delta):
|
def change_volume(self, delta):
|
||||||
self.volume = int(min(100, max(0, self.volume + delta)))
|
self.volume = int(min(100, max(0, self.volume + delta)))
|
||||||
if self.audio_device:
|
if self.audio_device:
|
||||||
try:
|
self._safe("Volume set", lambda: self.audio_device.setOutputVolume(self.volume))
|
||||||
self.audio_device.setOutputVolume(self.volume)
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Volume set error: {e}")
|
|
||||||
print(f"🔊 NAO volume: {self.volume}%")
|
print(f"🔊 NAO volume: {self.volume}%")
|
||||||
self.update_title()
|
self.update_title()
|
||||||
|
|
||||||
@@ -469,9 +461,13 @@ class NaoTeleop:
|
|||||||
time.sleep(1.0)
|
time.sleep(1.0)
|
||||||
self.motion.setMoveArmsEnabled(True, True)
|
self.motion.setMoveArmsEnabled(True, True)
|
||||||
|
|
||||||
|
print("🦶 Initializing walk engine...")
|
||||||
|
self._safe("moveInit", self.motion.moveInit)
|
||||||
|
print("🦶 Walk engine ready")
|
||||||
|
|
||||||
carpet_config = [["StepHeight", 0.028], ["MaxStepFrequency", 0.6], ["TorsoWy", 0.05]]
|
carpet_config = [["StepHeight", 0.028], ["MaxStepFrequency", 0.6], ["TorsoWy", 0.05]]
|
||||||
|
|
||||||
print("\n🎮 Controls: WASD/Arrows=Walk | IJKL=Head | SPACE=Reset head | 1=Wave | M=Music search (↑↓ select, Enter confirm) | P=Pause | X=Stop | ,/.=Seek | -/==Volume | T=TTS | ESC=Quit")
|
print("\n🎮 Controls: WASD/Arrows=Walk | IJKL=Head | SPACE=Reset head | 1=Wave | M=Music search (↑↓ select, Enter confirm) | P=Pause | X=Stop | -/==Volume | T=TTS | ESC=Quit")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
while self.running:
|
while self.running:
|
||||||
@@ -559,20 +555,13 @@ class NaoTeleop:
|
|||||||
self.music.pause()
|
self.music.pause()
|
||||||
elif event.key == pygame.K_x:
|
elif event.key == pygame.K_x:
|
||||||
self.music.stop()
|
self.music.stop()
|
||||||
elif event.key == pygame.K_COMMA:
|
|
||||||
self.music.seek(-10)
|
|
||||||
elif event.key == pygame.K_PERIOD:
|
|
||||||
self.music.seek(10)
|
|
||||||
elif event.key in (pygame.K_MINUS, pygame.K_KP_MINUS):
|
elif event.key in (pygame.K_MINUS, pygame.K_KP_MINUS):
|
||||||
self.change_volume(-5)
|
self.change_volume(-5)
|
||||||
elif event.key in (pygame.K_EQUALS, pygame.K_KP_PLUS):
|
elif event.key in (pygame.K_EQUALS, pygame.K_KP_PLUS):
|
||||||
self.change_volume(5)
|
self.change_volume(5)
|
||||||
elif event.key == pygame.K_t:
|
elif event.key == pygame.K_t:
|
||||||
self.typing_mode = True
|
self.typing_mode = True
|
||||||
try:
|
self._safe("stopMove", self.motion.stopMove)
|
||||||
self.motion.stopMove()
|
|
||||||
except Exception as e:
|
|
||||||
print(f"stopMove error: {e}")
|
|
||||||
|
|
||||||
if not self.typing_mode and not self.music_search_mode:
|
if not self.typing_mode and not self.music_search_mode:
|
||||||
keys = pygame.key.get_pressed()
|
keys = pygame.key.get_pressed()
|
||||||
@@ -663,27 +652,15 @@ class NaoTeleop:
|
|||||||
print(f"\u26a0\ufe0f Frame error (continuing): {e}")
|
print(f"\u26a0\ufe0f Frame error (continuing): {e}")
|
||||||
finally:
|
finally:
|
||||||
print("🛑 Shutting down...")
|
print("🛑 Shutting down...")
|
||||||
try:
|
self._safe("stopMove", self.motion.stopMove)
|
||||||
self.motion.stopMove()
|
self._safe("music stop", self.music.stop)
|
||||||
except Exception as e:
|
|
||||||
print(f"stopMove error: {e}")
|
|
||||||
try:
|
|
||||||
self.music.stop()
|
|
||||||
except Exception as e:
|
|
||||||
print(f"music stop error: {e}")
|
|
||||||
if self.audio_device:
|
if self.audio_device:
|
||||||
try: self.audio_device.unsubscribe(self.audio_service_name)
|
self._safe("audio unsubscribe", lambda: self.audio_device.unsubscribe(self.audio_service_name))
|
||||||
except Exception as e: print(f"audio unsubscribe error: {e}")
|
|
||||||
if getattr(self, "sound_receiver", None):
|
if getattr(self, "sound_receiver", None):
|
||||||
try: self.sound_receiver.close()
|
self._safe("sound receiver close", self.sound_receiver.close)
|
||||||
except Exception as e: print(f"sound receiver close error: {e}")
|
|
||||||
if hasattr(self, 'video') and self.video_client:
|
if hasattr(self, 'video') and self.video_client:
|
||||||
try: self.video.unsubscribe(self.video_client)
|
self._safe("video unsubscribe", lambda: self.video.unsubscribe(self.video_client))
|
||||||
except Exception as e: print(f"video unsubscribe error: {e}")
|
self._safe("rest", self.motion.rest)
|
||||||
try:
|
|
||||||
self.motion.rest()
|
|
||||||
except Exception as e:
|
|
||||||
print(f"rest error: {e}")
|
|
||||||
pygame.quit()
|
pygame.quit()
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+14
-5
@@ -1,5 +1,14 @@
|
|||||||
qi
|
# Install with: pip install -r requirements.txt
|
||||||
opencv-python
|
#
|
||||||
numpy
|
# NOTE: the NAOqi Python SDK ("qi") is NOT on PyPI. It ships separately from
|
||||||
pygame
|
# Aldebaran/SoftBank Robotics as a platform-specific wheel (pynaoqi-python*).
|
||||||
yt-dlp
|
# Download it from the Softbank Robotics / Aldebaran developer portal, match
|
||||||
|
# it to your Python version/OS, and install it with:
|
||||||
|
# pip install ./pynaoqi-*.whl
|
||||||
|
# See README.md for details.
|
||||||
|
|
||||||
|
opencv-python>=4.8
|
||||||
|
numpy>=1.24
|
||||||
|
pygame>=2.5
|
||||||
|
pyaudio>=0.2.13
|
||||||
|
yt-dlp>=2024.1.1
|
||||||
|
|||||||
Reference in New Issue
Block a user