Difference between revisions of "Info"
m |
|||
(One intermediate revision by the same user not shown) | |||
Line 62: | Line 62: | ||
** /home/nico/NICO-software/api/src/nicomotion/scripts/nicomotion/Motion.py | ** /home/nico/NICO-software/api/src/nicomotion/scripts/nicomotion/Motion.py | ||
** the one marked italics is the one that runs | ** the one marked italics is the one that runs | ||
* Controlling CoppeliaSim from Python: [https://github.com/stepjam/PyRep PyRep] | |||
=== ROS support and other related info === | === ROS support and other related info === | ||
Line 156: | Line 156: | ||
* Nico's ears contain microphones sampled with high-quality studio-recording equipment with 96 kHz, 24 bit sampling | * Nico's ears contain microphones sampled with high-quality studio-recording equipment with 96 kHz, 24 bit sampling | ||
* Nico's speaker | * Nico's speaker | ||
=== Controlling fans in the hand === | |||
* see [https://dai.fmph.uniba.sk/projects/nico/changing_fan_speed.txt info from Seed Robotics] | |||
__notoc__ | __notoc__ |
Latest revision as of 15:43, 9 January 2024
On this page, you can find all the available information about the robot - including various technical details etc.
Construction
The robot has 22 degrees of freedom including two 6DOF hands (the IDs on the left show the motor ID in the low-level communication protocol):
1 - right shoulder forward-backward 2 - left shoulder forward-backward 3 - right shoulder lift 4 - left shoulder lift 5 - right elbow 6 - left elbow 19 - head rotate 20 - head lift 21 - right shoulder left-right 22 - left shoulder left-right 30 - controller in the right hand with palm sensor 31 - right wrist rotate 33 - right wrist left-right 34 - right thumb lift 35 - right thumb close 36 - right index finger 37 - right other fingers 40 - controller in the left hand with palm sensor 41 - left wrist rotate 43 - left wrist left-right 44 - left thumb lift 45 - left thumb close 46 - left index finger 47 - left other fingers
Software, links documentation
- NICO Software github repository: https://github.com/knowledgetechnologyuhh/NICO-software
- the modifications for 6DOF hand are not available yet
- our local generated API documentation: https://kempelen.dai.fmph.uniba.sk/nico-api-doc/_build/html/index.html
- Pypot framework https://github.com/poppy-project/pypot
- SeedStudio manual https://kb.seedrobotics.com/doku.php?id=nico:start
- Dynamixel protocol documentation https://emanual.robotis.com/docs/en/dxl/protocol1/
- Dynamixel SDK https://emanual.robotis.com/docs/en/software/dynamixel/dynamixel_sdk/overview/
- working demos:
- first, run 'source api/activate.sh' from ~/NICO-software
- ~/NICO-software/api/examples/nicomotion/4-finger-nico/expressive_demo.py
- ~/NICO-software/api/examples/nicomotion/4-finger-nico/relax.py
- ~/NICO-software/api/examples/nicoface/*
- ~/NICO-software/api/examples/nicoaudio/*
- ~/direct-control/DynamixelSDK/c/example/protocol1.0/hands_demo/linux64/hands_demo
- even though the sources are distributed in different places, for instance:
- /home/nico/.NICO-python3/lib/python3.8/site-packages/nicomotion-1.0-py3.8.egg/nicomotion/Motion.py
- /home/nico/.NICO-python3/lib/python3.8/site-packages/nicomotion/Motion.py
- /home/nico/NICO-software/api/src/nicomotion/build/lib/nicomotion/Motion.py
- /home/nico/NICO-software/api/src/nicomotion/scripts/nicomotion/Motion.py
- the one marked italics is the one that runs
- Controlling CoppeliaSim from Python: PyRep
- from Connor Gäde at the University of Hamburg:
The current version was tested with ROS melodic and noetic. We have never tested ROS 2. Our setup checks for specific ROS Versions during installation, so it won’t build the ROS packages under ROS 2. ROS noetic natively uses python 3. ROS melodic officially only supports python 2.7, but our python 3 setup does make it use the python 3 executable of the virtual environment. This works for the examples provided, but might cause issues if you want to install additional ROS packages. Our setup already has to compile cv_bridge for python 3, other packages might have similar issues. The emotionrecognition module requires an old tensorflow version, so it will only run on python 3.7.X and earlier since only tensorflow 2 is supported from python 3.8 onwards. We are currently in the process of eliminating that dependency. The moveit integration hasn’t been updated in a while and won’t work with the noetic version since moveit broke backwards compatibility. There were also issues with moveit under python 3 melodic if I remember correctly. Other than that, everything should be python 3 compatible.
Low level control of motors and other
- connections PC to robot
- USB3 - left camera
- USB3 - right camera
- USB - dynamixel protocol RS485 bus control of all motors and palm sensors typically at /dev/ttyUSB0
- USB - control of LED matrix displays in the head, typically at /dev/ttyACM0 - these respond to text commands at 115200 bps
- USB - audio in (microphones)
- USB - audio out (speaker)
- optional: USB for programming controllers in the hands
- hands_demo.c - simple demo made by modifying an example of DynamixelSDK
- low-level motor configuration know-how from Pedro: configuring_motors.txt (all of them are switched to protocol 1 as of Oct 12th 2021)
- More about motor limits and values
Vision
NICO’s head features two parallel See3CAMCU135 cameras with 4K resolution (4,096 × 2,160). The cameras have an opening angle of 202◦. Via the API, the camera can be configured to transmit only parts of the image and thus constrain the field of view to a human opening angle of 70◦.
To simply test the view of the cameras (they create the video devices /dev/video0 - left camera, /dev/video2 - right camera), you can use:
fswebcam --device /dev/video0 imageL.jpg fswebcam --device /dev/video2 imageR.jpg
or to take pictures and record videos in interactive application:
cheese
News from April 2022
We have experimented with cameras without IR filter.
When used at daylight, they are missing the green channel, it appears grey.
Update (May 2022): This has been resolved by using lens with IR filter. See the comparison without and with IR filter:
The See3Cam CU135 has two output modes - MJPG and UYVY. It seems that MJPG works only after replugging the cameras to USB, the stream of JPEGs probably gets interrupted in the middle and cannot be put in sync in the second run.
For low-level reading of images using video4linux, one can obtain a pair of images from the camera and saving them to PNG files. The utility is in our nico's github: https://github.com/Robotics-DAI-FMFI-UK/nico/tree/main/stereo
We have made some shots: calibration and candy datasets that we obtained with that program can be inspected/mirror-downloaded here: https://kempelen.dai.fmph.uniba.sk/files/nico-datasets/ (warning: some of those are for the old lenses, which were replaced, use the calib_new_lenses/ for the new lenses)
Regarding the "official" software for Nico with python examples. Programs there that work with cameras fail to work. The reason is that our cameras have different IDs than those that the software accepts. In order to be able to run those examples, you have to change the file
/home/nico/.NICO-python3/lib/python3.8/site-packages/nicovision/VideoDevice.py
and add our cameras names:
NICO_EYES = { "left": ( ... "usb-e-con_systems_See3CAM_CU135_290E4E06-video-index0", ), "right": ( ... "usb-e-con_systems_See3CAM_CU135_2A2A5006-video-index0", ), }
and also in the file
/home/nico/NICO-software/api/examples/nicovision/../../../json/nico_vision_calibration_params.json
we had to add the cameras, for example by copying the parameters for those other cameras that are there (more proper would be to calibrate these cameras and find the parameters). But even if this is changed, some examples still do not want to work... as they seem to struggle with the UYVY format.
Audio
- Nico's ears contain microphones sampled with high-quality studio-recording equipment with 96 kHz, 24 bit sampling
- Nico's speaker
Controlling fans in the hand