I wrote about this dataset (or more particularly, the research that pieces of this dataset powered, such as SayCan and RT2. [0]
Unfortunately, the makers of the robot platforms and a number of the researchers that created this work was ousted from Google when they shut down Everyday Robotics; a victim of the Google leadership team being allergic towards anything that isn't ads.
Still, it's great that the dataset is opened for public use.
Actions generated for the dataset was either created through traditional reinforcement learning methods via simulation or behavioral cloning. Fun fact - the behavioral cloning dataset was generated by humans wearing VR goggles controlling simulated or real robotic arms via the controller to perform tasks for the robot to copy. Really cool way to build up a nice dataset of actions.
[0] https://hlfshell.ai/posts/llms-and-robotics-papers-2023/
Thanks!
My background is in robotics with an emphasis on software; graduated into the first of our many once-in-a-lifetime economic meltdowns in 08, so I did software instead of robotics for a bunch of years. Getting my Masters now to try to reorient the career back to robotics; past five years I was working on cloud infrastructure for robotics fleets, now hoping to get into robotic planning and software.
Now's an interesting time to get into it - I suspect there's going to be a lot of innovation in how we approach the problem and how we build these tools still.
reply