git clone <https://github.com/haritheja-e/robot-utility-models.git
>cd robot-utility-models
Open 2 terminal windows. On one side follow “Robot Server” instructions and on the other side follow “Imitation in Homes” instructions below.
This is run in the Hello Robot’s root pip environment (outside conda/mamba)
robot-server
folder
cd robot-server
pip install -r requirements.txt
python3 start_server.py camera=d405
python3 start_server.py camera=iphone
Download mamba with instructions from here, also shown below
Run the commands below in terminal and follow the instructions
curl -L -O "<https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$>(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh
Re-open shell and run mamba activate
if not already in the base environment
Enter the imitation-in-homes
folder
cd imitation-in-homes
Create environment
mamba env create -f conda_env.yaml
Edit configs/env_vars/env_vars.yaml
project_root
variable to your imitation-in-homes directoryLoad and run the desired policy (VQ-BeT):
python run.py task=door_opening
python run.py task=drawer_opening
python run.py task=reorientation
python run.py task=bag_pick_up
python run.py task=tissue_pick_up
Ensure the device you’re SSH-ing from and the robot are on the same network
Open the UI in your local browser at http://ROBOT_IP:7860
Follow the instructions in the UI to run the policy. See the video below for a quick example of using the UI:
Optionally: run Diffusion Policy by
python run.py --config-name=run_diffusion task=door_opening
python run.py --config-name=run_diffusion task=drawer_opening
python run.py --config-name=run_diffusion task=reorientation
python run.py --config-name=run_diffusion task=bag_pick_up
python run.py --config-name=run_diffusion task=tissue_pick_up
robot-server/configs/network/network.yaml
on your robot, set
host_address
to your robot’s IPremote_address
to your workstation’s IPpython3 start_server.py
run.py
on the workstation. In imitation-in-homes/configs/run_diffusion.yaml
on your workstation set
network.host
to your workstation’s IPnetwork.remote
to your robot’s IPpython run.py
command